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Abstract 

The  promise  of  providing  an  intuitive  and  efficient  infonnation  interface,  while 
allowing  the  warfighter  to  perfonn  other  critical  tasks  such  as  targeting  or  aircraft  control, 
has  led  to  the  growing  popularity  of  Helmet  Mounted  Displays  (HMDs),  especially  on 
combat  aircraft.  Though  design  and  selection  of  competing  systems  is  critical  to 
optimized  perfonnance  and  safety,  structured  methods  for  the  evaluation  and  selection  of 
HMDs  are  not  often  used  in  the  acquisition  community,  leaving  selection  among 
alternative  designs  to  the  judgment  of  subject  matter  experts.  However,  technical 
decision-making  has  been  shown  to  be  flawed  without  the  use  of  a  structured  decision 
analysis  framework,  which  can  help  to  overcome  narrow  focus,  potential  bias,  and  human 
error.  This  thesis  proposes  a  HMD  design  evaluation  framework  using  fundamental 
multi-level  perfonnance  objectives  to  assess  the  value  of  an  alternative.  Supported  by 
principles  of  Human  Systems  Integration  (HSI)  and  Value-Focused  Thinking,  the 
framework  can  be  used  by  decision  makers  to  create  infonned,  defendable  judgments  that 
strive  to  increase  system  performance  while  decreasing  maintenance  and  integration 
resources.  The  17-factor  framework  is  illustrated  through  application  on  two  possible 
technology  solutions  for  a  fixed-wing  fighter  platfonn.  Based  on  the  senior  decision 
maker  preferences  and  available  system  data,  Alternative  1  -  Scorpion  Helmet  Mounted 
Cueing  System,  scored  0.481  out  of  1.0,  which  was  9.3  percent  higher  than  Alternative  2 
-  Joint  Helmet  Mounted  Cueing  System.  The  preferred  solution  was  also  insensitive  to 
adjusting  the  primary  Human  Factors  objective  weight  by  71 . 1%.  This  research 
successfully  demonstrated  a  quantitative  method  for  assessing  helmet  mounted  display 
system  alternatives  that  incorporate  critical  Human  Systems  Integration  principles. 
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A  DESIGN  EVALUATION  FRAMEWORK  FOR  HELMET  MOUNTED 
DISPLAYS  IN  FIGHTER  AIRCRAFT 


I.  Introduction 


General  Issue 

Proper  design  and  integration  of  human-machine  interfaces  is  critical  to  optimize 
overall  system  perfonnance.  Technology  advancements  have  produced  systems  which 
have  the  ability  to  gather  and  display  more  information  to  the  human  user  than  her  or  she 
can  efficiently  process  to  make  timely  decisions.  This  often  results  in  high  workload  and 
complicated  multitasking  environments  which  can  be  overwhelming,  forcing  the  operator 
to  share  attention  resources  with  many  tasks  simultaneously,  degrading  the  robustness  of 
decision  processing.  Overcoming  these  issues  necessitates  an  interface  that  augments  the 
operator’s  abilities  without  inducing  additional  control  tasks  or  negative  physiological 
effects.  The  promise  of  providing  an  intuitive  and  rapid  interface  to  information,  while 
allowing  the  warfighter  to  perfonn  other  critical  tasks  such  has  handling  a  weapon  or 
maneuvering  a  vehicle  has  led  to  the  growing  popularity  of  helmet  mounted  displays 
across  the  military  landscape.  Though  attractive  as  a  human-machine  interface  solution, 
improper  design  and  integration  of  the  HMD  system  could  lead  to  undesirable  situations. 
Thus  it  is  critical  that  the  HMD  be  fit  for  the  intended  use  and  environment. 

However,  the  evaluation  of  HMD  solution  options  is  not  always  perfonned  in  an 
analytical,  repeatable  manner.  Important  considerations  like  human  factors,  display 
usability,  and  user  integration  are  often  taken  into  account  by  the  best  judgment  of  the 
decision  maker.  Subject  matter  experts  are  left  to  evaluate  systems  that  at  times  weren’t 
designed  for  the  specific  platfonn  and  mission  the  HMD  will  be  employed  in.  These 
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deficiencies  can  lead  to  a  system  that  does  not  meet  the  user’s  requirements  or  ultimately 
can  lead  to  mishaps,  increased  cost,  and  sub-optimal  perfonnance.  In  order  to  make  fair 
and  balanced  comparisons,  an  assessment  framework  is  needed  that  will  allow  decision 
makers  to  perform  robust  technology  analyses  on  helmet  mounted  displays,  based  upon 
the  value  the  technology  brings  to  the  overall  mission  objective.  This  thesis  discusses  the 
research  conducted  on  helmet  mounted  displays  and  its  input  into  an  evaluation 
framework,  which  could  be  used  to  support  HMD  integration  decisions  in  fighter  aircraft. 

Background 

In  modem  combat,  lean  and  agile  forces  have  become  the  core  of  the  United 
States  military.  In  order  to  maintain  combat  superiority,  advanced  technology  must 
augment  the  minimized  manpower.  Integrated  technology  multiplies  individual  and  unit 
capabilities,  allowing  warfighters  to  strategically  engage  larger  enemy  forces.  To  achieve 
the  best  perfonnance  and  gain  this  advantage,  system  design  and  implementation  must 
consist  of  careful  fusion  of  technology  with  user  capabilities  and  limitations.  One  such 
technology,  who’s  advantage  is  very  closely  linked  to  proper  human-technology 
integration  is  the  Helmet  Mounted  Display  (HMD).  Rapidly  growing  in  its  uses 
throughout  the  military,  the  HMD  seeks  to  enhance  the  user’s  overall  situation  awareness 
by  augmenting  human  attributes  like  vision,  audition,  and  system  control. 

HMD  Use  in  Military  Aviation 

Described  by  Melzer  and  Moffitt,  a  helmet  mounted  display  has  the  basic 
elements  of  “an  image  source  and  collimating  optics  in  a  head  mount”  (J.E.  Melzer, 

1997).  While  this  description  is  still  accurate  when  describing  modem  HMDs,  for 
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military  aviation  it  is  appropriate  to  also  include  a  tracking  system  that  is  often  used  to 
couple  the  position  or  orientation  of  the  head  or  eye  with  one  or  more  aircraft  systems. 
Figure  1  illustrates  the  basic  components  of  an  HMD  with  this  description. 


Figure  1  :  Basic  components  of  a  modern  HMD 


Primarily  used  to  provide  the  significant  amount  of  tactical  and  strategic 
information  critical  to  successful  engagements  in  the  modern  battlefield,  helmet  mounted 
displays  have  found  functionality  across  military  applications  as  they  can  present  this 
information  in  an  intuitive  “eyes  out”  fonnat.  Warfighters  can  maintain  enhanced 
awareness  without  taking  their  attention  off  the  very  dynamic  combat  setting.  This 
advantage  is  most  noticeable  in  the  military  aviation  community,  where  efficient 
execution  of  complex  tasks  in  a  high  workload  environment  necessitates  an  optimized 
user-machine  interface.  The  user’s  ability  to  rapidly  acquire,  comprehend,  and  act  on 
changing  infonnation  is  the  key  to  mission  effectiveness  and  survival.  An  HMD  can 
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increase  the  safety  and  survivability  of  a  pilot,  as  well  as  enhance  their  lethality  by 
allowing  line  of  sight  steering  of  weapons  and  sensors.  However,  if  consideration  is  not 
given  to  the  specific  user-environment  combination,  a  sub-optimal  HMD  can  negatively 
impact  safety  and  survivability  for  our  warfighters  by  over  saturating  attention  resources 
and  instigating  chronic  physiological  damage.  With  the  use  of  thorough  analytical  design 
and  evaluation  methods,  followed  by  intelligent  decision  making  practices,  these  negative 
attributes  can  be  avoided. 

Human  System  Integration 

Human-technology  integration  has  long  been  a  challenge,  due  to  the  variation  and 
often  unpredictability  of  the  user  populations.  Following  World  War  II,  experts  found 
that  combat  systems  were  at  times  unsafe  and  difficult  to  operate  because  the  human  user 
was  not  sufficiently  considered  in  the  development  process.  These  suboptimal  designs 
led  to  underperformance,  increases  in  mishaps,  as  well  as  higher  costs  in  training  and 
healthcare  for  the  users.  In  an  attempt  to  alleviate  these  issues  General  Max  Thurman, 
eventual  Vice  Chief  of  Staff  of  the  U.S  Army,  stressed  “We  must  quit  manning  the 
equipment  and  start  equipping  the  man”  (HPW/HPO).  Eventually,  in  the  1980’s,  the 
fonnal  Human  Systems  Integration  (HSI)  discipline  was  conceived.  By  considering  user 
concerns  such  as  safety,  human  factors,  training  and  survivability,  the  ultimate  goal  of 
HSI  practitioners  is  to  reduce  the  total  life  cycle  costs  of  the  system  while  maximizing 
overall  system  perfonnance  by  applying  human  centric  principles  throughout  the 
planning  and  development  phases.  This  holistic  design  and  management  approach  can  be 
applied  to  all  human  integrated  systems,  including  HMDs.  Without  effective  HSI,  the 
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program  can  become  a  victim  of  expensive  retrofits,  unsafe  conditions,  and  subsequent 
inflated  total  life-cycle  costs  (Booher,  2003). 

Problem  Statement 

It  is  extremely  important  that  design  decisions  are  examined  for  their  implication 
on  the  entire  integrated  system.  Too  often  development  programs  fail  to  consider  the 
human  and  their  associated  requirements  as  part  of  the  system  (HPW/HPO).  To  achieve 
sustainable  enhanced  performance,  the  system  needs  to  be  defined  by  more  than  just  the 
hardware  and  software  components,  the  operator  and  maintainer  must  be  included. 

There  has  been  a  recent  resurgence  of  HSI  efforts  and  it  is  becoming  more 
common  practice  to  intimately  include  HSI  principles  as  part  of  design  requirements  and 
key  perfonnance  parameters  (KPPs)  in  development  of  new  technology.  This  is  a 
successful  approach,  when  the  initial  product  design  can  be  centered  around  a  particular 
system  and  use.  But  how  can  we  ensure  a  previously  developed  piece  of  equipment, 
commercial  off  the  shelf  (COTS),  meets  the  specific  needs  of  the  user  in  a  sustainable, 
cost  effective  manner?  According  to  the  HSI  handbook,  “COTS  content  increases  the 
need  for  HSI  assessment  and  risk  mitigation”  (HPW/HPO).  Currently,  the  military 
acquisition  community  commonly  relies  on  the  judgment  of  decision  makers  and 
technical  subject  matter  experts  (SMEs)  and  their  assessments  of  the  proposed 
technology.  The  problem  with  this  method  is  that  it  can  involve  a  multitude  of  competing 
decision  criteria  and  the  analysis  can  miss  interactions  between  expertise  domains,  which 
can  negatively  impact  system  performance  and  affordability.  Additionally,  decisions  can 
be  made  by  parties  with  differing  priorities.  A  program  manager  may  have  different 
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primary  goals  (e.g.,  cost  and  schedule)  than  the  user  (e.g.,  effectiveness  and  safety)  for 
example,  which  could  impact  the  resulting  products. 

What  is  needed  is  a  design  evaluation  framework  that  maintains  focus  on 
pertinent  system  performance  objectives  and  assesses  the  technology’s  ability  to  meet 
these  objectives  in  a  methodical,  repeatable  manner.  Furthermore,  a  framework  should 
allow  for  decision  makers  to  perfonn  an  intelligent  trade  space  analysis.  This  research 
investigates  a  framework  that  uses  decision  analysis  techniques  to  perform  a  human 
centric  examination  of  design  alternatives,  applied  to  helmet  mounted  displays.  By 
perfonning  this  type  of  evaluation  on  material  solution  alternatives,  HSI  goals  of 
optimum  performance  and  minimized  total  life-cycle  costs  can  be  managed. 

Research  Question 

The  primary  research  question  that  will  be  the  basis  for  this  investigation  and  analysis 
effort  is: 

What  is  an  evaluation  framework  for  HMD  designs,  allowing  for  solution 
comparison  and  informed  decision  making? 

Investigative  Questions 

The  following  are  additional  investigative  questions,  used  to  guide  the  development  of  an 
evaluation  framework: 

What  are  the  major  objectives  for  HMD  use  in  a  fighter  aircraft? 

What  HMD  factors/attributes,  with  an  emphasis  on  HSI  principles,  are  important 
when  comparing  technology  solutions? 
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How  should  an  evaluation  framework  for  HMD  designs  be  created,  aggregating 

the  above  factors? 

How  should  the  factors  be  weighted? 

What  are  the  value  metrics  for  those  factors? 

How  sensitive  is  the  preferred  solution  to  framework  parameters? 

Methodology 

Decision  Analysis  techniques  are  used  on  a  daily  basis,  whether  performed 
formally  or  not.  If  one  were  to  buy  a  house,  they  might  analyze  factors  such  as  square 
footage,  style,  number  of  rooms,  property,  floor  plan,  and  price.  The  extent  of  the  list  and 
importance  of  each  factor  would  be  different  for  each  person.  One  may  value  yard  size 
as  the  most  important  characteristic  while  another  sees  quantity  of  bedrooms  as  the 
deciding  factor.  These  importance  metrics  may  be  based  on  the  decision  maker’s  current 
state  of  existence,  such  as  financial  resource  availability,  expected  timeline  for  owning 
the  home,  or  suitability  for  the  family  dynamic.  The  prospective  home  buyer  would  take 
all  these  aspects  into  account  before  making  a  decision  on  which  home  to  purchase. 
Additionally,  disagreements  and  inconsistencies  between  stakeholders  throughout  a 
decision  process  can  erode  efficiency  and  relationships.  It  is  important  to  practice  an 
analytical  approach  to  evaluation  of  technology  options,  so  as  to  assure  the  greatest 
possible  chance  of  choosing  the  best  fit  solution. 

Value  Focused  Thinking  (VFT)  is  a  method  of  decision  analysis  that  focuses  the 
decision  around  the  objectives  the  decision  maker  (DM)  hopes  to  achieve  and  the 
characteristics  of  those  objectives  that  bring  value  to  the  decision.  In  contrast  to 
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alternative  based  decision  making,  which  starts  with  alternative  solutions  and  seeks  to 
differentiate  them,  value  based  decisions  begin  with  values  and  objectives  before 
identifying  solutions.  This  is  the  overarching  technique  to  be  used  in  the  development  of 
a  design  evaluation  framework  that  will  allow  decision  makers  to  perform  a  more  robust 
trade  space  analysis  of  alternatives.  A  methodical  approach,  originally  developed  by 
Ralph  Keeney  and  later  illustrated  by  Mark  Shoviak  (Shoviak,  2001),  will  be  followed 
for  thorough  investigation  of  the  problem  and  alternative  analysis.  This  process  begins 
with  an  examination  of  the  problem  space  and  elicitation  of  system  objectives,  where 
levels  of  abstraction  will  be  addressed.  Objective  hierarchies  will  then  be  developed, 
eventually  determining  metrics  and  associated  functions  to  determine  how  much  value 
the  factor  brings  to  the  decision.  Factor  and  objective  weighting  will  help  specify  the 
solution  for  the  particular  decision  maker  and  an  example  analysis  will  be  performed  on 
technology  solution.  Finally  a  discussion  of  the  results  will  be  presented. 

Assumptions  and  Limitations 

The  research  and  conclusions  described  in  this  document  are  presented  with 
several  assumptions  and  limitations.  The  first  limitation  is  that  no  new  experimental  data 
collection  will  be  utilized.  Due  to  resource  availability  and  timeframe  considerations, 
new  human-technology  experimentation  is  not  advantageous.  Furthermore,  significant 
research  has  previously  been  perfonned,  providing  sufficient  amounts  of  data  for  the 
scope  of  this  effort.  Therefore,  past  studies  and  existing  data  sources  will  be  levied.  The 
second  assumption  is  with  regards  to  HMD  uses.  As  this  technology  has  been 
implemented  in  many  forms  and  environments,  the  focus  of  this  research  will  be  kept  to 
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HMD  integration  in  fixed-wing  fighter  aircraft.  Another  assumption  is  that  current 
USAF  human  factors  limits,  safety  margins,  along  with  military  standards,  requirements, 
and  handbooks  will  be  used  for  design  factor  criteria.  When  applicable,  currently 
accepted  assessment  methods  and  metrics  are  assumed  to  be  accurate  and  sufficient. 

Implications 

The  results  and  discussions  included  in  this  study  could  be  used  by  Department  of 
Defense  engineers  and  program  managers  to  enhance  the  analysis  of  HMD  design 
alternatives.  By  using  a  specialized,  analytical  process  it  is  believed  that  an  optimum 
technology-mission  fit  can  be  achieved.  Human  Systems  Integration  studies  have  shown 
that  when  systems  are  designed,  implemented,  and  managed  with  robust  user-centric 
principles,  performance  and  safety  will  increase  while  total  life  cycle  costs  will  decrease. 
Models  created  from  this  study  can  assist  in  assuring  the  initial  technology  selection  is  in 
line  with  systematic  HSI  practices  and  creates  consensus  on  priorities  between 
stakeholders.  Furthermore,  the  principles,  methods,  and  techniques  utilized  in  this  study 
could  serve  as  a  baseline  for  intelligent  decision  analysis  of  other  human  interface 
technologies. 

Preview 

The  four  additional  chapters  to  follow  will  provide  more  detail  in  the  research, 
analysis,  and  conclusions  of  this  effort.  In  Chapter  2,  a  review  will  be  provided  of  the 
investigative  research  accomplished  on  the  topics  of  helmet  mounted  displays,  Human 
Systems  Integration,  and  the  decision  analysis  methodology,  Value  Focused  Thinking. 
The  methodology  of  the  research  and  analysis  will  be  described  in  detail  in  Chapter  3, 
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while  Chapter  4  provides  results  of  the  assessment.  The  fourth  chapter  will  also  include  a 
discussion  of  the  results  and  their  implications.  Finally,  recommendations  and  possible 
future  research  opportunities  will  be  proposed  in  Chapter  5. 
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II.  Literature  Review 


Chapter  Overview 

Human  technology  interfaces  consist  of  complicated  physical  and  psychological 
relationships.  Efficient  representation,  acquisition,  and  utilization  of  information  must  be 
carefully  paired  with  intuitive  system  control.  Military  weapons  systems  increase  the 
need  for  optimization  of  these  relationships,  as  they  are  critical  in  life  and  death 
situations.  Applying  an  analytical  approach  to  alternative  selection  will  help  developers 
and  decision  makers  consider  all  relevant  factors  of  total  system  performance  to  achieve 
an  optimized  solution  and  avoid  costly  negative  life  cycle  outcomes. 

The  research  found  in  this  chapter  focuses  on  the  main  goal  of  developing  a  HMD 
design  evaluation  framework,  by  investigating  the  concepts,  technology,  and  analysis 
techniques  pertinent  to  the  topic.  The  Human  System  Integration  (HSI)  perspective  of 
implementing  human  centric  design  and  development  practices  to  increase  system 
performance  and  decrease  total  life  cycle  costs  will  be  described,  as  its  principles  will  be 
used  throughout  the  framework  development.  To  gain  a  thorough  understanding  of  the 
technology  under  consideration,  this  chapter  will  also  present  helmet  mounted  display 
system  design  considerations,  uses,  and  implications.  Finally,  the  decision  analysis 
approach  of  Value  Focused  Thinking  (VFT)  will  be  explained  in  detail,  laying  out  the 
method  to  be  used  for  technology  examination  and  framework  consideration. 

Human  System  Integration 

Technology  is  constantly  advancing  and  growing  in  complexity.  However, 
systems  have  yet  to  reach  a  point  where  the  human  can  be  removed  from  the  operational 
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chain.  Even,  so  called,  “Unmanned”  Aerial  Vehicles  (UAVs)  have  human  operators  that 
manage  and  monitor  its  operations  from  a  remote  control  station.  Therefore,  current 
system  designs  must  include  aspects  of  human  technology  interaction.  A  human’s 
propensity  to  make  mistakes,  mixed  with  a  technology  interface  not  optimized  for  the 
user  can  lead  to  expensive  redesigns,  increased  total  life-cycle  costs,  or  even  tragic  loss  of 
life  (Booher,  2003).  Though  design  focus  can  often  be  on  the  technology  capability 
advancement  side,  the  human  element  must  be  considered  a  critical  component  of  the 
system  if  one  wishes  to  avoid  these  negative  results. 

The  Human  Systems  Integration  (HSI)  philosophy  maintains  the  idea  that  not 
only  can  catastrophic  outcomes  be  avoided  by  perfonning  human  centric  development 
and  management  processes,  but  dramatic  increases  in  perfonnance  and  decreases  in  costs 
can  also  be  achieved  (Booher,  2003).  This  is  accomplished  by  initiating  a  comprehensive 
HSI  strategy  early  in  the  design  and  development  process  and  maintaining  metrics  and 
checkpoints  throughout  the  program. 

Human  Systems  Integration  is  an  all-inclusive  management  and  technical 
approach,  which  incorporates  various  functional  areas  to  address  the  possible  human 
element  impacts  throughout  the  total  lifecycle  of  the  system  (HPW/HPO).  These 
functional  areas,  or  domains  (listed  in  Table  1),  are  seemingly  independent  factors  of 
human  characteristics  and  perfonnance.  However,  understanding  and  controlling  their 
interactions  are  keys  to  successful  HSI  implementation.  The  Department  of  Defense 
began  adopting  HSI  principles  in  the  latel980’s  with  the  US  Army’s  creation  of  the 
Manpower  and  Personnel  Integration  (MANPRINT)  program  (Booher,  2003).  Training 
accidents  and  fratricide  incidents  magnified  the  need  for  Army  leaders  to  shift  the  focus 
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of  technology  developers  away  from  “equipment  only”  and  towards  the  “total  system” 
view,  which  mandated  that  soldier  perfonnance  and  limitations  be  critical  considerations 
in  weapon  system  design. 

Table  1  :  HSI  Domains 
HSI  Domains 

Manpower 

The  number  and  mix  of  personnel  authorized  and  available  to  train,  operate,  maintain,  and  support 
each  system  acquisition. 

Personnel 

The  human  aptitudes,  skills,  knowledge,  experience  levels,  and  abilities  required  to  operate,  maintain, 
and  support  the  system  at  the  time  it  is  fielded  and  throughout  its  lifecycle. 

Training 

The  instruction  and  resources  required  to  provide  personnel  with  requisite  knowledge,  skills,  and 
abilities  to  properly  operate,  maintain,  and  support  the  system. 

Human  Factors  Engineering 

The  comprehensive  integration  of  human  capabilities  and  limitations  into  system  design,  development, 
modification  and  evaluation  to  optimize  human-machine  performance  for  both  operation  and 
maintenance  of  a  system.  Designing  systems  that  require  minimal  manpower,  provide  effective 
training,  can  be  operated  and  maintained  by  users,  and  are  suitable  and  survivable. 

Environment 

Environmental  factors  concern  water,  air,  and  land  and  the  interrelationships  which  exist  among  and 
between  water,  air,  and  land  and  all  living  things. 

Safety 

Safety  factors  are  operational  characteristics  that  minimize  the  possibilities  for  accidents  or  mishaps  to 
operators  which  threaten  the  survival  of  the  system. 

Occupational  Health 

Occupational  Health  factors  are  design  features  that  minimize  risk  of  injury,  acute  and/or  chronic 
illness,  or  disability,  and/or  reduced  job  performance  of  personnel  who  operate,  maintain,  or  support 
the  system. 

Survivability 

The  characteristics  of  a  system  that  reduce  risk  of  fratricide,  detection,  and  the  probability  of  being 
attacked;  and  the  enable  the  crew  to  withstand  man-made  or  natural  hostile  environments  without 
aborting  the  mission  or  suffering  acute  and/or  chronic  illness,  disability,  or  death. 

Habitability 

Factors  of  living  and  working  conditions  that  are  necessary  to  sustain  the  morale,  safety,  health  and 
comfort  of  the  user  population  which  contribute  directly  to  personnel  effectiveness  and  mission 
accomplishment,  and  often  preclude  recruitment  and  retention  problems. 


24 


The  concept  of  Human  System  Integration  has  grown  and  transfonned  since  the 
Anny’s  initial  adoption.  The  Department  of  Defense  (DoD)  recognizes  that  HSI  is  an 
important  tenant  in  various  levels  of  its  acquisition  practices,  requiring  a  cross  functional 
approach  for  implementation.  Therefore  they  have  placed  special  emphasis  on  HSI  by 
listing  it  as  a  requirement  for  all  acquisition  programs,  declaring  in  DoD  Instruction 
5000.02,  “The  Program  Manager  will  plan  for  and  implement  human  system  integration 
(HSI)  beginning  early  in  the  acquisition  process  and  throughout  the  product  life  cycle” 
(Defense,  2013).  In  today’s  acquisition  programs,  HSI  principals  can  be  found  in  all 
phases  of  the  design  and  development  process.  Some  areas  of  influence  include  mission 
analysis  and  requirements  detennination,  generation  of  a  systems  engineering  plan, 
modeling  and  simulation,  design  working  groups,  test  and  evaluation,  upgrades,  and  even 
technology  retirement  and  disposal. 

In  a  system’s  life  cycle,  one  of  the  most  beneficial  times  to  include  HSI  practices 
is  during  requirements  writing  and  alternative  selection.  Booher  explains  that  most  life- 
cycle  costs  are  detennined  by  decisions  made  early  in  system  development  and  it  is  at 
this  phase  that  human-related  problems  can  be  most  easily  and  cost  effectively  addressed 
(Booher,  2003).  It  is  important,  especially  with  COTS  items,  to  not  only  create 
requirements  with  human-centric  attributes,  but  to  have  evaluation  measures  that  help 
determine  the  degree  to  which  alternatives  meet  those  requirements.  It  is  this  background 
infonnation  and  underlying  purpose  that  will  drive  the  construction  of  the  HMD 
evaluation  framework  at  the  center  of  this  research  topic. 
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Helmet  Mounted  Displays 


HMD  Use  in  Military  Aviation 

One  of  the  earliest  combat  aviation  uses  of  a  HMD  system  was  in  the  1970s  as  a 
control  device  for  the  gimbaled  gun  on  the  U.S.  Anny’s  AH-1G  Huey  Cobra  attack 
helicopter  (Clarence  E.  Rash,  2000).  The  system  was  designed  to  allow  weapon  aiming 
by  simply  using  the  pilot’s  line  of  sight  to  place  the  helmet-mounted  reticle  over  the 
intended  target.  After  the  Army’s  initial  success,  the  U.S.  Navy  found  use  for  this  head 
tracking  technology  in  their  F-4  Phantom  fixed-wing  jet  aircraft  as  a  means  for  the  pilot 
to  interface  with  the  fire-control  radar.  Cueing  discretes  and  a  visor-projected  reticle 
allowed  for  daytime,  off-boresight,  air-to-air  targeting  of  the  AIM-9H  Sidewinder 
missile.  In  both  of  these  cases  it  was  found  that  the  HMD  yielded  a  significant  reduction 
in  the  time  required  to  engage  weapons  on  a  target  (Clarence  E.  Rash,  2000).  As  with 
most  technology,  these  simple  beginnings  gave  life  to  development  of  more  capabilities 
and  integration  possibilities  between  aircraft  and  users.  As  seen  on  platforms  such  as  the 
Air  Force  A-10,  F-16,  and  F-35,  the  modern  HMD  not  only  allows  for  weapons  cueing 
but  aircraft  monitoring,  threat  detection,  and  enhanced  synthetic  vision.  If  integrated 
appropriately,  an  HMD  has  the  ability  to  reduce  the  cognitive  demands  of  aircraft 
monitoring,  while  simultaneously  allowing  the  pilot  timely  and  effective  acquisition  of 
enemies. 

HMD  Basics 

As  described  in  the  introduction,  a  modem  combat  aviation  HMD  primarily 
consists  of  four  components:  the  mounting  platform,  image  source,  relay  optics,  and 
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tracker  subsystem.  There  are  a  variety  of  ways  these  components  are  designed  and 
integrated,  depending  on  the  intended  use  and  available  technology. 

Traditionally  serving  as  a  protection  device  for  the  aviator,  the  helmet  is  typically 
used  as  the  mounting  platform,  hence  the  moniker  “Helmet-Mounted  Display”. 
Depending  on  the  vendor,  modem  HMDs  have  either  included  a  new  helmet  as  part  of 
the  system,  such  as  the  Joint  Strike  Fighter  HMD,  or  have  been  designed  as  retrofits  to 
the  current  helmets  in  use,  like  the  Scorpion  Helmet  Mounted  Cueing  System. 

Regardless  of  the  design,  the  helmet  mounting  platfonn  must  continue  to  not  only  protect 
the  user  but  also  provide  a  stable  base  for  the  display  and  tracker.  Misalignment  between 
the  display  and  user’s  eye  caused  by  slippage  or  vibrations  can  make  the  image  look 
distorted,  degrading  the  utility  of  the  system,  or  even  inducing  physiological  issues  such 
as  nausea,  headaches,  and  dizziness  (Clarence  E.  Rash,  2000). 

The  image  source  is  the  display  device  used  to  produce  the  symbology  or  sensor 
video  feed  that  will  be  provided  to  the  user  via  relay  optics.  Early  HMDs  used  cathode- 
ray-tubes  (CRTs)  or  image  intensification  tubes  as  the  image  source.  However, 
advancing  technology  such  as  liquid  crystal  displays  (LCDs)  and  light  emitting  diodes 
(LEDs)  have  allowed  for  higher  brightness  levels,  addressability  and  image  contrast 
while  drawing  less  power  and  adding  less  weight  to  the  system. 

The  relay  optics  often  consist  of  a  combination  of  lenses  for  magnification  and 
collimation  of  the  image.  Magnification  is  often  necessary  as  the  display  is  typically 
formed  on  a  small  substrate  having  a  diagonal  less  than  2  cm.  Collimation  is  important  to 
project  the  image  near  optical  infinity  so  the  operator  does  not  have  to  accommodate 
differently  when  reading  the  display  information  or  looking  at  distant  objects.  The  optics 
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can  either  form  an  image  on  a  reflector  in  front  of  the  eye,  precluding  the  operator  from 
seeing  real-world  objects  or  form  the  image  on  a  combiner  in  front  of  the  pilot’s  eye, 
overlaying  the  displayed  image  onto  the  natural  scene.  The  later  design  permits  the  user 
to  “see  through”  the  display,  enabling  the  user  to  fall  back  to  the  use  of  the  natural  scene 
when  the  display  fails.  The  combiner  is  typically  integrated  into  the  helmet  visor  itself  or 
as  a  separate  piece  mounted  between  the  visor  and  the  user’s  face. 

The  requirement  for  a  head  tracker  is  dependent  on  the  intended  use  of  the  HMD 
system.  If  utilized  simply  as  an  information  status  display,  with  no  need  for  spatially- 
referenced  symbology,  then  the  head  tracker  would  provide  no  added  benefit  to  the 
wearer.  However,  HMDs  have  found  much  of  their  combat  advantage  comes  from  the 
ability  to  integrate  with  aircraft  avionics  and  weapons,  allowing  for  a  “point  and  shoot” 
enemy  engagement  along  with  persistent  aircraft  information  or  navigation  aids.  To 
provide  this  capability,  the  system  must  know  where  the  pilot  is  looking,  requiring  the 
inclusion  of  a  head  tracking  device.  Additionally,  advancements  in  sensors  have  created 
synthetic  imagery  such  as  forward  looking  infrared  (FLIR)  short  wave  infrared  (SWIR), 
which  can  be  displayed  in  front  of  the  pilot’s  eye,  providing  an  enhanced  view  of  the 
environment.  This  capability  may  also  require  a  head  position  triangulation  if  the  sensor 
is  mounted  elsewhere  in  the  aircraft.  There  are  many  technologies  currently  fielded  or  in 
development  that  can  be  used  to  track  the  helmet  position  and  orientation.  These  include; 
magnetic,  optical,  inertial,  acoustic,  or  a  hybrid  combination  of  techniques.  Selection  of  a 
tracker  option  must  be  given  special  consideration  as  the  technology  may  have  an  impact 
on  other  aircraft  systems.  Also,  the  tracker  system  has  a  considerable  influence  on 
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latency  affects,  which  impact  the  usability  and  acceptability  of  the  HMD  system  (Randall 
E.  Bailey,  2004). 

It  is  the  technology  selection  and  integration  for  these  four  basic  components, 
along  with  the  processor  that  interfaces  the  HMD  with  the  aircraft,  which  set  HMD 
systems  apart  from  each  other.  Designers  make  trades  and  sacrifices  to  create  the  best 
possible  solution  for  the  combat  aviator.  The  following  paragraphs  will  address  some  of 
the  uses,  concerns  regarding  HMDs,  and  how  design  considerations  strive  to 
accommodate  them. 

HMD  Uses  in  a  Fighter  Aircraft 

Helmet  mounted  displays  are  finding  value  in  many  different  environments  and  in 
serving  many  different  purposes,  to  include  training,  inspections,  gaming,  fire-fighting, 
surgical  aids,  computer-aided  design,  and  remotely-piloted  vehicles  (Clarence  E.  Rash, 
2000).  However,  the  most  use  and  advancement  of  HMD  technology  seems  to  be  in  the 
field  of  military  aviation,  as  it  can  bring  enhancements  to  both  the  pilot’s  situational 
awareness  (SA)  and  enemy  engagements. 

A  basic  definition  of  situational  awareness  is;  the  user’s  ability  to  perceive 
elements  in  their  environments,  comprehend  the  meaning  of  these  elements,  and  predict 
their  near  future  status  (Endsley,  1995).  For  all  pilots,  attaining  and  preserving  SA  means 
continual  cross  checks  of  safety  of  flight  instrumentation  while  maintaining  external 
visibility  for  obstacles  and  aircraft  avoidance,  communicating  with  other  aircraft  and 
control  stations,  deciphering  maps  for  navigation,  and  employing  emergency  checklists 
when  necessary  in  order  to  perceive  the  current  status  of  the  aircraft.  Further  the  pilot 
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must  interpret  this  infonnation  and  project  potential  future  events  to  enable  rapid  decision 
making  when  time  critical  events  occur.  The  combat  aviation  environment  not  only 
requires  these  elements  for  SA  but  add  the  additional  complications  of  high  performance 
maneuvers  and  strategic  engagements;  which  call  for  awareness  of  infonnation  such  as 
weapons  management,  enemy  identification  and  tracking,  multi-aircraft  coordination,  and 
other  tactical  information.  A  pilot’s  ability  to  readily  gather  this  information  from 
multiple  sources  is  the  key  to  maintaining  high  situation  awareness. 

As  aircraft  perfonnance  increased,  the  short-comings  of  head  down  displays 
became  apparent.  Faster  and  more  maneuverable  aircraft  meant  a  more  rapidly  changing 
environment.  Even  the  brief  time  it  took  for  pilots  to  look  down  at  their  displays  to  gain 
necessary  ownship  and  environment  infonnation  could  actually  degrade  their 
environmental  SA.  The  development  of  head  up  displays  (HUDs)  gave  the  pilot  critical 
flight  data  in  an  eyes-out  fonnat.  These  see-through  combiners,  fixed  directly  in  front  of 
the  pilot’s  forward  view,  display  ownship  infonnation  such  as  attitude,  altitude,  airspeed, 
and  heading  along  with  tactical  information  such  as  tracking  cues  and  warnings. 

Though  the  HUD  offers  many  benefits,  and  can  be  found  on  almost  all  high 
perfonnance  military  aircraft,  its  limitation  is  its  fixed  position.  Covering  between  15-20 
degrees  of  the  forward  field  of  view,  the  pilot  is  still  required  to  take  their  eyes  away 
from  possible  off-boresight  targets,  obstructions,  or  navigation  points  to  gain  ownship 
and  tactical  information.  The  helmet  mounted  display  takes  the  eyes  out  enhanced  SA 
advantage  of  the  HUD  and  provides  it  across  the  entire  viewing  range  of  the  wearer.  By 
having  the  flight  critical  information  persistently  within  the  pilot’s  line  of  sight,  overall 
performance  and  operational  safety  is  ideally  increased  (Clarence  E.  Rash,  2000). 
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The  same  enhancement  found  for  overall  situational  awareness  can  also  be  seen 


for  the  tactical  envelope  of  an  HMD  equipped  aircraft.  Currently,  pilots  are  required  to 
look  head  down  at  multiple  displays  to  gather  2-D  positional  and  status  information  about 
the  rapidly  changing  combat  environment.  As  their  attention  is  transferred  back  outside 
the  aircraft,  they  must  then  translate  the  information  to  a  3-D  space  relative  to  their 
position.  This  infonnation  synthesis  takes  time  and  cognitive  workload,  reducing  the 
pilot’s  reaction  time  to  possible  threats.  The  importance  of  this  mapping  process  cannot 
be  underestimated  as  the  USAF  has  lost  over  101  lives  and  65  aircraft  during  the  past  21 
years  due  to  pilot  loss  of  spatial  orientation,  a  problem  that  is  particularly  prevalent  in 
fighter  and  rotor  wing  aircraft  (Miller,  2014)  With  the  advent  of  the  helmet  mounted 
display,  the  potential  exists  to  assemble  the  appropriate  infonnation  for  the  pilot  and  to 
automatically  display  it  in  a  manner  that  streamlines  data  gathering  and  assembly, 
pennitting  faster  decisions,  enabling  more  rapid  and  informed  responses  to  external 
events. 

Target  engagement  has  also  been  revolutionized  by  the  HMD.  Since  the 
boresight  mounted  machine  guns  of  World  War  I,  fighter  pilots  have  been  cuing  weapons 
by  pointing  the  nose  of  the  aircraft  in  the  direction  of  the  target.  Propulsion  and 
maneuverability  have  been  key  components  to  survival  in  this  combat  dynamic,  a 
philosophy  that  was  unchanged  until  fourth  generation  missiles  appeared  (Clarence  E. 
Rash,  2000).  These  air-to-air  weapons,  capable  of  more  than  50G  turns,  could  be 
released  on  targets  at  very  large  off-boresight  angles.  Without  an  HMD,  aircraft  radars 
and  sensors  are  used  to  cue  missile  seeker  heads,  often  requiring  the  pilot  to  maneuver 
into  a  position  that  may  not  be  tactically  advantageous.  HMDs  can  be  coupled  to  the 
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missile  seeker  or  aircraft  sensor  to  allow  for  a  “look  and  shoot”  capability,  which  can 
allow  the  pilot  to  cue  and  fire  a  missile  without  having  to  give  up  a  position  of  advantage 
(C.  Arbak,  1988).  By  using  the  HMD  as  a  cuing  tool,  inferior  aircraft  armed  with  GEN-4 
weapons  have  a  distinct  advantage  in  an  air-to-air  engagement,  where  aircraft 
performance  is  now  outweighed  by  the  speed  and  accuracy  at  which  a  missile  can  be 
deployed. 

HMD  Concerns  and  Design  Considerations 

As  with  any  human  machine  interface,  helmet  mounted  display  designs  must  take 
into  account  not  only  technology  perfonnance  but  also  how  that  technology  interacts  with 
human  characteristics  and  limitations.  There  are  physiological  and  psychological 
concerns  that  can  make  the  system  unusable  or  even  endanger  the  operator  if  not 
considered.  The  primary  design  factors  for  see-through  HMDs  will  be  discussed  here. 

Weight,  Center  of  Gravity,  &  Overall  Size 

The  head  borne  weight  and  center  of  gravity  (C.G.)  of  the  HMD  is  extremely 
important  to  the  design  of  the  system.  In  high  perfonnance,  ejection  seat  aircraft  these 
characteristic  have  a  large  impact  on  operator  fatigue  and  safety.  Any  weight  deviation 
from  the  normal  C.G  of  the  user’s  head  will  cause  additional  muscle  strain  and  tend  to 
fatigue  the  pilot,  especially  during  high  g  maneuvers.  In  addition,  overall  weight  could 
affect  the  chances  of  survival  during  an  ejection  event  as  the  neck  is  exposed  to 
extremely  high  loads  during  the  catapult  phase  and  as  the  seat  reaches  the  wind  stream. 
The  slam-back  effect  caused  by  the  sudden  blast  of  air,  and  exacerbated  forces  with  the 
extra  weight,  can  be  catastrophic.  Further,  added  weight  has  the  potential  to  induce 
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operator  fatigue,  especially  when  the  helmet-mounted  display  is  worn  throughout  the 
entire  mission  and  potentially  could  add  to  chronic  musculoskeletal  injury  over  years  of 
use. 

The  overall  size  of  the  helmet  portion  is  also  important,  as  head  space  is  often 
limited  in  a  fighter  aircraft.  The  display  must  be  small  enough  to  allow  the  pilot  to 
perform  full  scanning  patterns,  including  the  check  six  maneuver  (turning  to  look  behind 
the  aircraft).  Advances  in  materials  and  electronics  have  greatly  reduced  these  hazards, 
but  not  to  the  point  of  completely  eliminating  them. 

Ocularity  and  Field  of  View 

Ocularity  refers  to  whether  the  HMD  displays  imagery  in  a  monocular,  binocular, 
or  biocular  manner,  defined  by: 

Monocular  -  single  image  source,  viewed  by  a  single  eye 
Binocular  -  each  eye  views  an  independent  image  source 
Biocular  -  single  image  source,  viewed  by  both  eyes 
(Clarence  E.  Rash,  2000) 

The  ocularity  of  a  device  should  be  determined  by  how  it  is  to  be  used,  as  a  tradeoffs 
must  be  made  with  the  additional  weight,  optical  alignments,  visual  rivalry  issues  and 
power  consumption  that  multiple  or  single  image  sources  would  require.  Table  2  shows 
some  of  the  advantages  and  disadvantages  between  the  ocularity  options. 
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Table  2  :  HMD  Ocularity  (Clarence  E.  Rash,  2000) 


Ocularity 

Benefits 

Downside 

Monocular 

•  Minimum  weight 

•  Less  stringent  alignment 

•  Typically  least  expensive 

•  Eye  without  display  continues  to 
sample  outside  world 

•  Possible  visual  rivalry 
between  eyes 

•  Possible  asymmetric  C.G. 

•  Smaller  F.O.V 

•  No  ability  for  stereoscopic 
display 

Binocular 

•  Can  provide  stereo  viewing 

•  More  symmetric  C.G 

•  Wider  F.O.V.  than  monocular 

•  No  ocular  rivalry  between  eyes 

•  Heaviest  weight 

•  More  complex  alignment  and 
adjustments 

•  Most  expensive 

Biocular 

•  Wider  F.O.V.  than  monoculor 

•  No  ocular  rivalry  between  eyes 

•  Potentially  less  expensive  and 
lighter  than  binocular 

•  Heavier  than  monocular 

•  Typically  reduced  luminance 

•  Heavier  and  more  complex 
alignment  than  monocular 

Display  Field  of  View  (F.O.V)  describes  the  size  of  the  apparent  image  space, 
more  formally  defined  as  the  maximum  image  angle  of  view  that  can  be  seen  through  an 
optical  device  (Clarence  E.  Rash,  2000).  As  a  reference,  the  total  binocular  F.O.V  of  the 
human  visual  system  is  about  200°  horizontal  by  130°  vertical.  The  F.O.V  required  for  a 
HMD  is  highly  dependent  on  intended  usage  and  input  symbology/imagery.  A  wider 
F.O.V  is  useful  to  create  an  immersive  visual  environment  but  will  add  extra  wieght  to 
the  display.  If  the  display  is  to  be  used  for  weapons  cueing  and  a  minimum  amount  of 
information  communication,  a  smaller  F.O.V  is  accetpable. 

Display  Resolution 

An  important  aspect  of  detennining  the  “sharpness”  of  the  imagery  of  a  display 
can  be  described  as  the  resolution.  A  low  resolution  image  will  lack  detail  and  may 
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appear  blurry.  Typically,  this  metric  can  be  described  as  the  pixel  count  in  a  display, 
width  x  height.  For  example,  current  high-definition  liquid  crystal  display  (LCD)  and 
light  emitting  diode  (LED)  televisions  are  said  to  have  a  resolution  of  1080  or  1920 
pixels  wide  by  1080  pixels  tall.  However,  the  size  of  the  display  can  have  an  effect  on 
the  clarity  of  the  image  as  the  pixels  are  often  scaled  to  keep  the  same  count.  Therefore, 
the  more  accurate  definition  of  resolution  would  be  the  number  of  pixels  (or  lines)  that 
can  be  adequately  distinguished  across  the  scene  (Information  Display  Measurements 
Standard,  vl.03,  2012).  For  this  effort,  pixel  density,  or  pixel  per  degree  of  vision,  will 
be  the  metric  used  to  help  define  the  HMDs  resolution. 

Display  Brightness  and  Contrast  for  See-Through  Displays 

The  challenge  with  see-through  displays,  such  as  the  HUD  or  HMD,  is  designing 
a  combiner  with  acceptable  transmission  qualities  while  still  providing  an  image  with 
high  contrast  when  viewed  against  a  high  luminance  ambient  scene.  This  type  of  HMD 
necessitates  a  higher  brightness  image  source  to  maintain  a  contrast  that  allows  the 
display  to  be  legible  under  all  ambient  (environmental)  conditions.  This  concept  is 
shown  in  Figure  2,  where  ‘L’  represents  luminance  or  brightness  (measured  in  foot- 
Lamberts),  ‘T’  represents  transmittance  or  the  amount  of  light  allowed  through  the 
medium  (measured  in  percent  of  incoming  light),  and  ‘R’  represents  reflectance  or  the 
amount  of  light  bouncing  off  the  medium  (measured  in  percent  of  incoming  light). 
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Figure  2  :  Affects  on  Display  Brightness 


The  contrast  is  calculated  by  the  ratio  between  the  brightest  part  of  the  display  and  the 
darkest.  In  the  case  of  a  see-through  display,  the  darkest  part  would  be  the  luminance  of 
the  ambient  scene  observed  by  the  pilot  (Lc)  and  the  brightest  would  be  that  luminance 
plus  the  luminance  added  by  the  image  source  (LHmd)-  This  ratio  is  shown  below  in 
Equation  1. 


CR  = 


Lhmd  Lp 

Ln 


Eq.  1. 


Where 


Lhmd  ~  x  T()  x  R( 


L0=LaxTaxTvxTc 


Eq.  2. 
Eq.  3. 


And  where 

Li  is  the  luminance  of  the  image  source 
To  is  the  transmittance  of  the  optics 
Rc  is  the  reflectance  of  the  combiner 
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Tc  is  the  transmittance  of  the  combiner 


La  is  the  luminance  of  the  ambient  scene 
Ta  is  the  transmittance  of  the  canopy 
Ty  is  the  transmittance  of  the  visor 

The  transmittance  and  reflectance  characteristics  of  the  visor  and  other  HMD  components 
are  carefully  calculated  to  maintain  a  sufficient  contrast  ratio  in  all  possible 
environments. 

System  Latency  and  Accuracy 

System  latency,  for  the  purposes  of  this  research,  refers  to  the  flawed  alignment  of 
the  symbology  overlaid  on  the  real  world  as  the  user  moves  their  head  and  occurs  as  a 
result  of  the  time  delay  from  sensor  input  to  display  projection.  On  an  aircraft,  this 
latency  is  contributed  to  by  not  only  all  components  of  the  HMD  system  and  the 
communication  between  them  but  the  systems  of  the  aircraft  platform  that  feed 
information  to  the  HMD  system.  The  HMD  system  contributors  are;  the  time  to 
detennine  head  position,  which  is  communicated  to  the  display  processor,  then  the  time 
to  process  the  data  and  compute  the  new  graphic,  which  then  creates  the  new  image  and 
communicates  to  the  image  display  which  then  refreshes  to  render  the  new  image  on  the 
combiner.  Each  component  function  adds  to  the  total  latency  time  and  can  cause  nausea 
and  other  physiological  issues  if  not  minimized.  Advanced  processing  and  refresh  rates 
have  significantly  reduced  the  total  system  latency  in  HMDs  but  can  still  be  a  problem  as 
no  systems  have  yet  to  reach  true  zero  latency. 
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Accuracy,  is  similar  to  latency  in  that  it  is  a  measure  of  deviation  in  symbology 
location  relative  to  real  world  references.  However,  latency  is  used  to  express  a  time 
delay  in  symoblogy  placement  or  a  lag  found  during  head  motion,  while  accuracy  can  be 
thought  of  as  the  difference  in  a  static  scene.  Very  intuitively,  if  there  is  a  significant 
amount  of  inaccuracy  in  the  system,  operations  and  safety  can  suffer  severe  detriments, 
especially  in  a  crowded  airspace  environment  where  it  is  important  to  delineate  between 
types  of  air  traffic.  A  system’s  accuracy  is  dependent  on  the  tracker  type  being  used  and 
the  processing  power  of  the  system. 

Display  Symbology  and  Clutter 

As  with  any  cockpit  displays,  the  selection  of  symbology  or  imagery  to  be 
displayed  is  as  important  as  the  technical  specifications  that  describe  the  image  source. 
For  the  military,  symoblogy  shapes,  colors,  and  mechanics  are  standardized  and  can  be 
found  in  documents  such  as  MIL-STD-1472,  Human  Engineering  or  MIL-STD-2525 
Common  Warfighting  Symbology,  and  others.  Issues  like  unstandardized  symbology 
sets,  over  clutter  of  information,  and  lack  of  intuitive  mechanization  can  cause  serious 
problems,  including  loss  of  situational  awareness,  task  saturation,  and  negative  training. 
Designers  of  modern  HMDs  have  accounted  for  this  by  allowing  symbol  customization. 
The  customer  is  then  responsible  for  determining  symbology  schemas,  using  sound 
human  factors  practices  for  display  design. 

User  Acceptance 

All  of  the  above  factors  combine  to  create  the  user  acceptance  aspect.  An  HMD 
system  must  be  usable  by  many  body  sizes  and  personalities  for  long  periods  of  time. 
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Any  features  that  can  possibly  cause  discomfort  or  affect  efficient  usage  could  cause  the 
system  to  be  unacceptable.  Building  in  user  adjustability,  such  as  selectability  of  display 
position,  brightness  settings,  and  declutter  options  are  often  necessary  to  account  for  the 
variation  in  pilot  population  and  uses. 

Below  in  Table  3  is  a  summary  of  some  of  the  HMD  systems  that  have  been 
developed  in  the  past  four  decades. 

Table  3  :  Selection  of  fixed-wing  HMD  programs  (Clarence  E.  Rash,  2000) 


Time  Frame 

System  Name 

Developer 

Aircraft  Platform 

1970s 

Display  And  Sight  Helmet  (DASH) 
series 

Elbit  Systems  Ltd 

Fielded  on  Israeli  F-15,  F-16, 
F/A-18C 

Romanian  Mig-21 

1980s 

Agile  Eye 

Kaiser  Electronics 

Unfielded;  Experimental 

System 

1990s 

Viper  series 

GEC-Marconi 

Avionics  Ltd 

Unfielded;  Experimentally 
flown  on  F-16,  AV-8B, 

Tornado 

1990s 

TopSight/TopNight 

Thales  Avionics 

Fielded  on  Mirage  and  Rafale 

1990s 

Joint  Helmet  Mounted  Cuing 

System  (JHMCS) 

Vision  Systems 
International  (VSI) 

Fielded  on  F-15,  F-16,  F/A-18 

2000s 

Scorpion  Helmet  Mounted  Cueing 
System  (HMCS) 

Gentex  Corporation 

Fielded  on  F-16,  A-10 

2000s 

Helmet  Mounted  Display  System 
(HMDS) 

Vision  Systems 
International  (VSI) 

In  development  for  F-35 

2000s 

Striker  HMD 

BAE  Systems 

Fielded  on  Eurofighter 

Typhoon  and  Gripen 
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Decision  Analysis  -  Value  Focused  Thinking 

Decision  analysis,  at  its  core,  is  a  logical,  repeatable  approach  to  illuminating  best 
solutions  to  complicated  problems.  At  the  end  of  an  analysis,  the  decision  maker  is 
provided  with  the  infonnation  necessary  to  make  the  most  educated  quantified  judgment 
of  the  alternatives. 

One  method  for  applying  the  decision  analysis  process  is  called  Value  Focused 
Thinking.  Introduced  by  Ralph  Keeney,  this  approach  consists  of  two  overarching 
activities;  first  deciding  what  you  want  and  then  figuring  out  how  to  get  it  (Keeney, 

1992).  While  it  sounds  simple  and  intuitive,  Keeney  points  out  that  we  often  do  not 
follow  this  logic.  In  fact  we  have  learned  most  of  our  decision  making  practices  based 
upon  how  decisions  have  been  posed  to  us  throughout  our  lives.  For  example,  would  you 
like  to  wear  shorts  or  long  pants?  Would  you  like  to  write  with  a  pen  or  a  pencil?  This 
method  of  starting  with  what’s  available  and  taking  the  best  from  your  choices  is  known 
as  Alternative-Focused  thinking  (Keeney,  1992).  While  this  may  be  the  “natural”  way  of 
making  a  decision,  in  complex  situations  it  can  limit  the  understanding  of  the  problem 
space  and  lead  to  negative  unforeseen  consequences.  Value-Focused  Thinking  (VFT), 
while  more  difficult  and  time  consuming,  is  meant  to  thoroughly  examine  the  full 
problem  context  and  uncover  objectives  or  opportunities  not  readily  apparent  to  the 
decision  maker.  By  starting  with  a  problem  and  first  determining  important  aspects  of 
the  decision  that  bring  value  to  the  decision  maker,  one  can  avoid  inadequate  fixes  that 
can  stem  from  impetuously  attempting  to  force-fit  a  solution. 

Value-focused  thinking  involves  defining  the  decision  context  and  objectives,  or 
something  that  one  desires  to  achieve.  Both  Parnell  and  Keeney  recognize  the 
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delineation  of  two  types  of  objectives,  fundamental  and  means.  Fundamental  objectives 
are  those  that  represent  the  end  goal  of  a  decision  while  the  means  objectives  describe 
how  the  fundamental  objectives  will  be  met  (Gregory  S.  Parnell,  2013).  These 
fundamental  objectives  are  often  missed  in  alternative-focused  thinking,  by  jumping 
straight  to  means  objectives  and  limiting  the  solution  options  to  be  examined.  As 
illustrated  in  Figure  3,  if  only  means  objectives  are  identified  one  can  miss  solutions  that 
may  more  fully  meet  their  higher  level  fundamental  objectives. 


Fundamental  Objectives 


Figure  3  :  Alternatives  in  problem  space 

Shoviak  (2001)  has  broken  down  the  concepts  of  value-focused  thinking  into  a  ten 
step  model  development  process  that  helps  to  define  the  decision  space  by  eliciting  the 
objectives  and  developing  values  with  associated  metrics.  An  analysis  can  then  be 
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performed  to  determine  a  best-fit  solution,  ultimately  giving  the  senior  decision  maker 
the  infonnation  needed  to  make  an  informed  decision.  This  process  is  laid  out  in  Figure 
4. 


Step  10  :  Conclusions  & 
Recommendations 


Figure  4  :  Value-Focused  Thinking  Process 


Step  1 :  Problem  Identification 

The  goal  of  this  first  step  is  to  develop  a  well-defined  statement  of  the  problem 
that  ensures  complete  understanding  of  the  decision.  Characterizing  this  decision  frame  is 
key  to  understanding  and  determining  the  important  factors,  objectives,  and  decision 
boundaries.  This  process  can  be  accomplished  through  interviews  with  the  senior 
decision  maker,  user  community,  and  other  stakeholders.  Questions  should  be  proposed 
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with  the  purpose  of  eliciting  the  infonnation  that  will  make  clear  the  possible  courses  of 
action  that  may  be  considered  and  which  may  not  (Gregory  S.  Parnell,  2013).  This  step 
will  be  the  basis  for  which  objectives  and  values  are  created,  then  further  translated  into 
metrics  and  alternative  analysis.  The  frame  will  also  help  to  focus  the  analyst  and 
decision  makers  to  a  certain  scope.  Inadequate  problem  identification  or  decision 
framing  can  cause  a  failure  of  the  end  goal  by  proposing  a  solution  that  meets  the  defined 
problem  but  doesn’t  solve  the  REAL  WORLD  issue. 

Step  2  :  Create  Objectives  Hierarchy 

In  this  next  step,  objectives  important  to  the  decsion  makers  are  elicited  from 
relevent  stakeholders  and  used  to  construct  a  hierarchy  that  will  be  decomposed  into 
something  that  is  complete,  measurable,  mutually  exclusive,  and  concise.  To  be 
considered  complete,  the  hierarchy  must  include  all  relevant  factors  for  the  decision 
frame.  These  factors,  or  objectives,  are  used  for  evaluating  the  desirablity  of  eventual 
alternatives  or  consequences  of  a  decision.  It  is  at  this  point  the  first  prominent 
difference  between  alternative-based  thinking  and  value-focused  thinking  can  be  seen. 
The  objectives,  and  not  the  available  alternatives,  should  be  the  driving  force  behind 
decision  making.  One  should  begin  at  the  highest  level  objective,  or  strategic  objective, 
and  decompose  into  objectives  that  bring  value  to  the  high  level.  For  example,  if  your 
startegic  objective  is  to  ‘buy  the  best  home  for  your  family’,  you  might  decompose  that 
objective  into  lower,  operation  level  objectives  to  consider  such  as:  ‘size  of  house’, 
‘location’,  ‘property’,  etc.  Those  would  then  evantually  be  broken  out  again  into 
objectives,  known  as  the  tactical  level,  where  specific  values  can  be  measured  and 
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weighted.  Examples  of  these  lowest  level  values  would  be  ‘square  footage’,  ‘distance  to 
schools  or  parks’,  ‘yard  size’,  etc.  When  moving  down  the  hierachy,  each  subsequent 
level  should  answer  the  question  “What  do  you  mean  by  that?”  about  the  preceeding 
level.  By  following  this  decomposition  hierarchal  process  a  complete  examination  of  the 
objectives  and  values  can  be  accomplished.  To  meet  VFT  requirements  these  lowest 
level  attributes  must  be  measurable,  whether  through  a  natural,  constructed,  or  proxy 
method.  They  must  also  be  mututally  exclusive  in  that  they  are  independent  of  each  other 
and  so  changing  the  value  of  one  does  not  in  turn  change  the  value  of  another.  The 
ultimate  goal  of  this  step  is  to  create  this  complete  hierarchy  in  a  concise  manner  to 
facilitate  calculations  and  communications  with  the  senior  decision  maker. 

Step  3  :  Develop  Evaluation  Measures 

After  completing  the  objectives  hierarchy,  the  next  step  is  to  detennine  evaluation 
measures  (metrics)  for  the  lowest  level  objectives.  These  metrics  are  meant  to  be 
quantitative  scales  that  will  eventually  be  the  basis  for  determining  how  much  value  the 
attribute  brings  to  the  decision  space.  The  measures  can  be  classified  as  natural, 
constructed,  or  proxy  (Keeney,  1992).  It  is  important  to  recognize  the  type  of  metric  that 
is  being  used,  as  it  can  add  or  subtract  validity  to  the  findings.  Natural  measures  are 
those  with  common  interpretations  or  uses,  as  in  using  ‘number  of  accidents’  for  a 
measure  of  the  objective  “minimize  the  accidents  at  an  intersection”.  These  are  the  most 
desirable  measures  for  a  decision  analysis  process  as  they  are  widely  accepted  and  are 
conducive  with  ease  of  communication.  Sometimes  a  natural  metric  doesn’t  exist  for  a 
given  objective.  In  this  case  one  might  construct  a  measure  that  is  specific  for  the 


44 


decision  context.  For  the  objective  “maximize  the  local  acceptance  of  a  new  store”  there 
is  not  a  natural  metric.  However,  one  can  be  constructed  that  assigns  a  scaled  measure 
for  the  ‘number  of  groups  that  publicly  oppose  the  new  store’.  The  final  type  of  metric  is 
known  as  a  Proxy  measure.  This  is  a  metric  that  indirectly  indicates  the  impact  on  an 
objective  and  should  only  be  used  if  a  natural  or  constructed  measure  cannot  be 
detennined.  Regardless  of  the  type  of  measure  used,  the  scale  should  be  clear  and 
meaningful  to  the  decision  maker.  It  is  also  important  to  remember  to  choose  the 
appropriate  scale  bounds  for  the  decision  context.  If  an  alternative  arises  that  is  not  within 
the  bounds  then  it  cannot  be  compared  to  other  options  without  rescaling  the  values. 

Step  4  :  Create  Value  Functions 

At  this  point  it  can  be  seen  that  the  evaluation  measures  developed  in  the  last  step 
will  be  a  collection  of  different  units  and  scales,  which  will  prohibit  direct  comparison 
and  summations  to  obtain  a  total  score.  To  transform  these  evaluation  measures  into  a 
common  ‘unit  of  value’  a  value  function  is  developed  for  each  objective.  This  function 
can  be  represented  on  a  coordinate  plane  with  one  axis  labeled  as  the  metric  and  the  other 
as  the  perceived  value,  which  is  scaled  from  least  value  (0)  to  max  value  (1).  The  plotted 
function  should  be  shown  as  monotonically  increasing  or  monotonically  decreasing 
(Gregory  S.  Parnell,  2013).  In  other  words  as  the  objective  measure  increases  the  value 
will  always  be  either  increasing  or  decreasing.  Furthermore,  the  slope  of  these  trend  lines 
would  represent  a  change  in  the  rate  of  return.  In  the  monotonically  increasing  Figure  5, 
Function  1  would  represent  a  decision  maker  who  believes  the  rate  of  return  gradually 
decreases  as  the  evaluation  measure  is  increased.  Alternatively,  Function  2  represents  an 
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increase  in  the  expected  rate  of  rate  of  return  as  the  measure  increases.  Function  3,  shows 
a  decision  maker  who  believes  in  a  constant  rate  of  return  throughout  the  range  of  the 
bounded  evaluation  measure.  An  S-Curve  (Function  4)  is  used  when  there  are  inflection 
points  or  changes  in  rate  of  return  at  specific  values  of  the  measure.  This  can  often 
represent  an  “optimum  point”.  Piecewise  linear  forms  of  these  curves  types  are  also 
common  (Function  5),  if  there  are  known  points  with  unchanging  value  rates  between 
them. 


Figure  5  :  Value  function  types 

When  the  evaluation  measures  have  a  small  number  of  discrete  points  that  add  value,  a 
bar  graph  type  plot  can  be  used.  Occasionally  value  functions  that  are  not  monotonic  can 
be  found,  i.e.,  they  rise  then  fall  or  vice  versa.  This  often  happens  when  two  evaluation 
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measures  are  combined  instead  of  being  kept  separate  (Gregory  S.  Parnell,  2013).  For 
example,  the  value  of  the  number  of  bedrooms  in  a  house  may  increase  up  to  4  bedrooms 
then  decreases.  This  decrease  could  represent  the  decision  maker’s  belief  that  there 
would  be  too  many  rooms  to  clean  or  it  would  drive  an  increase  in  utility  bills.  While 
these  are  legitimate  trade-space  points,  it  is  better  practice  to  keep  the  evaluation  measure 
independent  of  each  other  (Gregory  S.  Parnell,  2013). 

When  creating  these  value  functions,  one  method  is  to  start  with  a  linear 
(increasing  or  decreasing)  trend  line  based  upon  current  knowledge  of  the  decision  maker 
and  other  stakeholders’  feelings  toward  the  metric.  Any  deviation  to  the  trend  line,  such 
as  creating  a  concave,  convex,  s-curve  or  asymptotic  shape  would  follow  an  elicitation 
from  the  decision  maker  or  subject  matter  expert  that  rationalizes  a  belief  in  a  change  in 
the  rate  of  return  for  that  particular  metric. 

Step  5  :  Weight  the  Objectives  Hierarchy 

While  the  objectives  hierarchy  created  in  step  2  illustrates  all  the  relevant 
objectives,  it  must  be  recognized  that  not  all  of  them  are  equally  important  to  the  decision 
maker.  To  create  an  accurate  model,  the  decision  maker’s  preference  between  them  must 
be  taken  into  account.  By  determining  a  weighting  value  for  each  objective,  the  relative 
importance  of  each  can  be  represented. 

One  technique  for  assigning  weights  is  the  direct  weighting  method.  In  this 
technique  the  decision  maker  directly  assesses  the  importance  of  one  objective  over 
another  independently,  that  is  without  consideration  as  to  how  much  the  individual 
objective  will  contribute  to  the  overall  score  (Shoviak,  2001).  Using  Figure  6  as  an 
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example,  one  would  start  at  the  lowest  level  objectives  (C)  of  the  hierarchy,  creating  local 
weights  for  C 1  and  C2  against  each  other,  and  C3  and  C4  likewise.  This  is  done  by 
asking  the  decision  maker  which  of  these  objectives  is  the  least  important.  The  indicated 
objective  would  be  assigned  a  variable,  X.  Then  the  decision  maker  would  be  asked  for  a 
value  that  represents  how  much  more  important  the  next  one  is  than  the  one  assigned 
with  variable  X.  This  question  is  repeated  for  each  of  the  values  in  the  connected  branch 
and  creates  an  algebraic  expression  that  is  set  equal  to  one.  After  solving  for  X,  each 
value  has  a  unique  local  weight,  or  w  as  it  will  be  labeled  in  Equation  4.  This 
methodology  is  used  on  each  of  the  lowest  objective  branches,  working  up  one  level  at  a 
time  until  every  objective  has  a  weight  relative  to  the  others  in  the  same  level. 


X  +  4X  =  1  X  +  3X  =  1 

X  =  0.20  X  =  0.25 


0.20 


Figure  6:  Example  Hierarchy  and  Weighting  Method 
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The  following  equation  (Eq.  4)  is  then  used  to  create  a  value  measure,  based  on  the  value 
function  curve  and  weight  described  above,  for  each  of  the  objectives. 

»  Eq.  4. 

r(j)  =  2>,.v,(x:) 

i=l 

Where, 

V (x)  is  the  alternative’s  value  of  x  objective 

X  =  {xl,x2,x3,...,xn) 

vf (x,.)  is  the  value  function  for  a  given  evaluation  metric  x, 

u’  is  the  weight  for  a  given  value  measure 

Step  6  :  Alternative  Generation 

After  following  the  VFT  unique  method  of  identifying  the  objectives  and  values 
first,  it  is  at  this  point  where  alternatives  are  brought  forward.  Generating  a  good  set  of 
alternatives  is  important  to  any  decision  making  process  as  it  can  severely  affect  the 
outcome.  While  this  may  sound  obvious,  there  are  specific  characteristics  of  an 
alternative  set  that  determine  if  it  is  adequate,  such  as  being  feasible,  complete, 
compelling  and  should  strive  for  diversity  in  the  options  (Gregory  S.  Parnell,  2013).  The 
feasible  term  should  apply  to  the  final  set  to  be  evaluated;  each  alternative  must  be  a 
course  of  action  that  can  be  taken.  In  other  words,  a  barrier  cannot  exist  that  would  make 
it  impossible  to  purse  one  or  more  of  the  alternatives,  such  as  a  non-negotiable 
timetable/cost  constraint.  However,  alternative  generation  should  be  an  iterative  process, 
so  it  is  often  helpful  to  ignore  constraints  in  the  early  brainstonning  phases  as  sometimes 
initial  constraints  can  be  relaxed  (Gregory  S.  Parnell,  2013).  A  complete  set  of 
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alternatives  has  a  full  description  of  each  option.  Parnell  gives  the  example,  “if  a 
decision  is  to  be  made  on  how  to  deploy  a  military  battalion  in  an  exercise,  each 
alternative  should  specify  mission  tasking  for  every  unit  within  the  battalion”  (Gregory  S. 
Parnell,  2013).  Another  important  characteristic  of  the  alternative  generation  step  is  each 
option  should  have  something  that  makes  it  appealing,  or  compelling,  over  at  least  one 
other  choice.  Kirkwood  uses  the  term  dominance  to  explain  this  concept  (Kirkwood, 
1997).  Alternative  A  dominates  alternative  B  if  all  the  attributes  of  A  are  at  least  as 
preferred  as  all  those  in  B ,  with  at  least  one  attribute  of  A  being  more  preferred.  If  an 
alternative  is  dominated  by  another,  according  to  this  description,  it  can  be  removed  as  an 
option.  An  alternative  set  that  meets  these  characteristics,  and  is  as  diverse  as  the 
problem  space  allows,  should  make  for  a  thorough  analysis  and  provide  the  decision 
maker  valuable  information. 

Step  7 :  Alternative  Scoring 

As  the  decision  model  has  been  developed  and  the  possible  alternatives  have  been 
derived,  it  is  at  this  point  that  the  evaluation  measures  are  populated  with  the  alternative 
characteristics.  Each  alternative  is  assessed  on  the  value  function  scales  created  in  step  4. 
Depending  on  the  metrics,  this  could  be  a  very  simply  “plug  and  chug”  process  or  could 
involve  user  evaluation  and  feedback.  The  end  goal  of  this  step  is  to  have  a  value 
measure  for  all  relevant  aspects  of  all  the  alternatives. 

Step  8  :  Deterministic  Analysis 

At  this  point,  the  Decision  Model  has  been  created  and  the  required  data  from  the 
alternatives  has  been  input.  The  deterministic  analysis  takes  the  inputs  from  step  7  and 
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calculates  the  global  value  score  for  each  alternative,  producing  a  number  that  can  be  a 
point  of  initial  comparison  between  them.  There  are  several  mathematical  functions  that 
can  be  used  to  perform  this  evaluation.  The  simplest  is  the  additive  value  function,  which 
assumes  mutual  preferential  independence  (Gregory  S.  Parnell,  2013).  In  other  words, 
this  method  can  be  used  if  the  value  function  of  one  evaluation  measure  does  not  depend 
on  the  level  of  another.  The  equation  presented  in  Step  5  of  this  section  is  used  by  the 
additive  value  function  method  to  calculate  the  aggregate  value  of  multiple  objectives. 

Step  9  :  Sensitivity  Analysis 

Once  the  global  values  are  determined  for  each  alternative  it  may  provide 
valuable  insight  to  the  decision  maker  to  see  how  the  alternative  rankings  would  change 
if  particular  data  in  the  model  was  adjusted,  essentially  determining  the  sensitivity  of  the 
preferred  alternative  to  a  change  of  assumptions  (Gregory  S.  Parnell,  2013).  This  can  be 
done  by  varying  the  weights,  value  curves,  or  evaluation  measures  for  the  alternatives.  It 
is  important  to  remember  that  when  changing  the  weighting  numbers,  always  insure  that 
they  sum  to  1.  Changing  evaluation  measures  and/or  weights  can  provide  essential 
information  if  the  alternative  design  or  environment  influence  is  subject  to  change  in  the 
decision  process.  For  example,  it  is  possible  that  a  senior  decision  maker  responsible  for 
choosing  a  new  air  to  ground  ordinance  may  want  to  see  how  the  preferred  alternative 
might  change  if  one  of  the  competing  contractors  assured  a  new  targeting  sensor 
technology  could  be  integrated  during  the  development  of  their  product.  This  would 
mean  changing  the  evaluation  measures  for  the  effected  objectives,  such  as  accuracy. 

The  decision  maker  might  also  be  curious  about  alternative  rankings  if  a  changing  fiscal 
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environment  drove  a  change  in  tactics  that  meant  this  new  ordinance  only  needed  to  be 
deployed  by  one  type  of  aircraft.  The  DM  could  go  into  the  model  and  alter  the  weight 
(or  relative  importance)  of  the  ‘achieve  multi-platform  integration’  objective. 

Step  10  :  Conclusions  and  Recommendations 

The  final  step  of  any  decision  analysis  process  is  to  provide  conclusions  and 
recommendations  to  the  decision  maker  and  stakeholders.  The  fonnat  of  this 
presentation  will  vary  depending  on  what  the  DM  has  requested,  but  generally  the  analyst 
will  gather  necessary  data  and  illustrations  to  clearly  communicate  their  findings  and  any 
insights  gained  during  the  analysis  process. 

Summary 

This  chapter  has  provided  infonnation  pertinent  to  the  research  and  methodology, 
including  the  topics  of  human  system  integration  (HSI),  helmet  mounted  displays,  and 
value-focused  thinking.  While  proven  to  be  important,  HSI  still  faces  challenges  in  its 
implementation  and  practices.  One  challenge  is  simply  resistance  to  change  in  the 
acquisition  community.  Additions  of  new  practices  and  checklists  are  seen  as  extra 
workload  instead  of  a  truly  beneficial  philosophy  incorporation.  This  can  only  be 
overcome  by  support  from  leadership  and  HSI  concept  training  for  the  workforce. 

The  other  important  challenge  has  to  do  with  the  inherently  difficult  nature  of 
incorporating  a  human  into  a  technology  system.  Robust  evaluations  frameworks  are 
necessary  to  improve  this  relationship,  so  the  HSI  principles  of  human  centric  design  will 
be  prevalent  throughout  this  decision  analysis  process  as  objectives  and  values  are 
defined. 
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The  description  of  HMD  technology,  their  human  impacts,  and  the  discussion  of 
current  systems  have  provided  context  for  the  alternatives  and  characteristics  to  be 
evaluated.  The  modem  warfighter  is  the  most  technologically  advanced  in  history.  Not 
only  have  the  weapon  systems,  like  fighter  aircraft  become  more  complex,  but  the  tactics 
they  employ  have  been  revolutionized  by  the  HMD.  This  critical  technology  requires  a 
robust  evaluation  methodology  to  assist  in  solution  selection  decisions  so  as  to  gain  the 
best  possible  combat  advantage. 

The  Value-Focused  Thinking  (VFT)  decision  analysis  process,  applied  in 
Chapters  3  and  4,  can  provide  an  analytical,  repeatable  assessment  based  on  the  value  a 
HMD  alternative  brings  to  the  overall  objectives. 
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III.  Methodology 


Chapter  Overview 

Chapter  III  employs  the  Value-Focused  Thinking  method  of  decision  analysis 
described  in  Chapter  II  to  create  an  evaluation  framework  for  HMDs.  This  chapter  will 
focus  on  the  left  side  of  the  VFT  “V”  (shown  below  in  Figure  7)  and  begins  with  problem 
identification,  followed  by  the  development  and  thought  processes  behind  construction  of 
the  objective  hierarchy.  Assignment  of  evaluation  measures  and  creation  of  their 
associated  value  functions  will  then  be  explained.  Decision  maker  inputs  will  then  be 
used  to  establish  weighting  values  for  the  objectives  presented  in  the  hierarchy.  When 
completed,  these  guidelines  and  measures  can  be  used  to  robustly  determine  an  optimum 
HMD  solution  for  a  fighter  aircraft  platform. 


Figure  7  :  VFT  Process  as  it  will  be  applied 
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Step  1  :  Problem  Identification 

Uniquely  difficult  to  evaluate  in  a  robust  manner  is  the  field  of  human  factors 
and  human  systems  integration.  Much  of  these  types  of  evaluations  are  conducted  with 
qualitative  human  subject  feedback  and  standards  that  cannot  account  for  the  entirety  of 
the  user  population,  explore  the  impact  of  a  small  number  of  factors  while  constraining 
all  other  factors  to  a  fixed  value,  and  often  aren’t  tailored  to  the  system’s  specific 
operational  environment.  While  a  recent  emphasis  on  HSI  has  brought  a  renewed  focus 
to  the  issues  of  human  centric  design  in  new  large  development  efforts  and  future 
programs,  there  is  still  the  issue  of  how  to  make  interface  design  decisions  in  a 
quantitative,  repeatable  manner.  Further  exacerbating  the  problem  is  the  push  to  use 
more  commercial  of  the  shelf  (COTS)  equipment  for  modernization  efforts  of  legacy 
aircraft  and  to  include  a  more  diverse  user  population.  COTS  items  are  developed  by 
commercial  industry  companies  to  be  useable  by  a  variety  of  platfonns  and  mission 
environments,  in  an  attempt  to  gain  the  most  business  with  a  diverse  customer  pool.  This 
approach  poses  a  problem  in  the  human  factors  realm  because  optimizing  the  technology- 
user  relationship  necessitates  building  a  system  to  the  specific  user-mission  combination, 
which  often  requires  tailoring  for  very  unique  attributes  not  found  across  multiple 
platforms.  As  stated  in  the  HSI  Handbook,  the  human  interface  of  COTS  products  are 
not  usually  customized  for  the  unique  operational  needs  of  the  Air  Force,  including  the 
critical  nature  of  many  tasks  (HPW/HPO).  To  accommodate  the  use  of  COTS  equipment 
into  a  unique  military  environment  it  is  imperative  that  the  evaluation  and  selection 
process  of  these  items  be  as  tailored  and  robust  as  possible.  Current  methods  for  early 
technology  down-selection  and  evaluation  generally  involve  subject  matter  expert 
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development  of  requirements,  followed  by  user  feedback  of  the  possible  system 
solutions.  While  this  technique  has  been  used  for  many  years,  it  is  susceptible  to  bias 
errors  and  inadvertent  omission  of  important  factors,  leading  to  non-optimized 
perfonnance  and  increased  costs  to  sustain  the  system.  So,  to  formalize  the  problem 
statement;  the  DoD  lacks  a  robust,  objective  method  for  human  centric  evaluation  of 
COTS  HMD  equipment  alternatives.  The  efforts  of  this  research  will  focus  around  the 
integration  of  a  helmet  mounted  display  onto  a  legacy  lighter  platfonn,  proposing  a  user 
centric  system  analysis  process  with  the  purpose  of  allowing  robust  and  repeatable 
decision  making  in  technology  selection. 

As  the  primary  purpose  of  this  research  is  to  demonstrate  a  quantifiable  method 
for  a  COTS  HMD  selection  process,  the  Value-Focused  Thinking  method  of  decision 
analysis  will  be  followed  for  the  remaining  9  steps  using  the  hypothetical  problem  that  an 
Air  Force  Acquisition  Fighter  Aircraft  Program  Office  needs  to  make  a  decision  on 
which  HMD  system  to  integrate  into  their  aircraft.  Step  1  of  the  VFT  process  is  problem 
identification  and  as  described  in  Chapter  II,  it  involves  the  fonnalization  of  the  problem 
space,  boundaries,  and  assumptions. 

Problem:  A  fixed-wing  fighter  platfonn  needs  a  helmet  mounted  display  solution 
to  provide  an  advantage  in  its  primary  roles  of  air-to-air  and  air-to-ground  combat.  The 
System  Program  Office  (SPO)  has  been  tasked  with  detennining  an  optimum  technology 
solution,  but  no  robust  evaluation  framework  exists  to  support  decision-making. 

Boundaries:  Due  to  fiscal  constraints,  the  development  of  a  new,  unique  HMD 
system  for  this  platfonn  is  not  an  option.  Commercial  off  the  shelf  (COTS)  products 
must  be  used  to  the  greatest  extent  possible. 
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Assumptions:  It  is  assumed  that  minimum  performance  requirements  have  been 
developed  by  the  user  command.  All  alternatives  to  be  examined  have  met  these 
requirements  and  so  will  be  compared  for  their  added  value  beyond  the  minimums.  Also, 
programmatic  topics  such  as  funding  and  schedule  have  been  determined  and  will  be 
addressed  separately  outside  of  this  analysis. 

Step  2  :  Create  Objectives  Hierarchy 

To  paraphrase  the  HSI  Handbook,  the  goal  of  human  systems  integration  is  to 
acquire  optimized  systems  that  enhance  performance  and  reduce  life  cycle  costs.  So  the 
goal  of  the  objective  hierarchy  for  this  effort  was  to  determine  what  factors  and 
relationships  would  impact  the  HMD  system’s  ability  to  optimize  performance  and 
operational  effectiveness  of  the  weapon  system.  The  first  realization  was  that  there  are 
multiple  levels  of  system  performance  that  can  be  optimized,  as  shown  below  in  Figure  8. 
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Not  all  of  these  levels,  such  as  ‘Aircraft  in  Mission  Space”,  can  be  quantitatively 
evaluated  during  an  early  COTS  equipment  selection  process.  Consequently,  technology 
evaluation  is  often  done  in  a  bottom-up  approach,  evaluating  attributes  in  the  lower  two 
levels  and  assuming  they  impact  the  upper  levels  of  performance  in  a  proportional 
fashion.  The  methodology  used  for  this  effort  was  to  take  a  user-centric  point  of  view 
and  start  at  the  top,  working  down  to  determine  exactly  what  measurable  technology 
characteristics  are  important  to  the  overall  goal  of  a  HMD.  This  was  accomplished  by 
asking  stakeholders  such  as  pilots,  engineers,  maintainers,  and  program  managers 
questions  like  “What  is  the  ultimate  goal  of  a  helmet  mounted  display  addition  to  the 
weapon  system?”  The  responses  are  presented  and  explained  below. 

Increase  Operational  Effectiveness 

This  objective  is  a  measure  of  the  impact  the  system  brings  to  the  combat 
environment.  Fighter  aircraft  serve  a  critical  role  in  the  military  airspace  and 
fulfill  missions  such  as  air  interdiction,  close  air  supports  (CAS),  offensive  and 
defensive  air-to-air  as  well  as  air-to-ground  engagements.  The  fundamental  goal 
of  a  capability  addition,  such  as  a  HMD,  should  be  to  have  a  positive  effect  on  all 
aircraft  responsibilities. 

Increase  Lethality  and  Survivability 

The  primary  role  of  a  military  combat  aircraft  is  to  engage  in  high  risk 
scenarios  where  survival  is  often  dependent  on  the  ability  to  destroy  the  enemy 
target  in  a  timely  and  effective  manner.  In  a  fighter  aircraft,  this  can  mean  being 
the  first  to  deploy  weapon  loads  and  avoiding  imminent  threats.  Capability 
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additions  should  strive  to  enhance  the  platforms  ability  to  strike  first  and 
accurately,  and  provide  defensive  awareness  for  increased  chances  of  survival. 
Increase  Safety 

An  additional  capability  or  system  should,  at  a  minimum,  avoid  increasing 
a  safety  risk.  The  possibilities  for  accidents  or  mishaps  to  operators  should  be 
carefully  addressed  and  controlled.  The  goal  of  many  aircraft  additions,  in 
particular  pilot  vehicle  interfaces  such  as  HMDs,  is  to  reduce  safety  hazards  in 
both  high  risk  and  benign  environments  by  offering  additional  situation  awareness 
so  as  to  avoid  obstacles  and  assist  in  basic  flight. 

This  type  of  question  would  be  followed  up  with  “In  order  to  accomplish  these 
fundamental  objectives,  what  means  objectives  must  a  helmet  mounted  display  meet?” 
Enhance  Situational  Awareness  (SA) 

An  aircraft  interface,  such  as  a  display,  should  seek  to  clearly  and 
accurately  provide  situation  information  in  a  manner  conducive  to  the  Perceive, 
Comprehend,  and  Prediction  of  future  state  goal  of  situational  awareness.  For 
fighter  pilots,  general  SA  can  be  decomposed  into  administrative  and  tactical 
situational  awareness.  Administrative  SA  can  be  thought  of  as  anytime  in  the 
sortie  when  the  pilot  is  not  engaging  an  enemy.  While  tactical  SA  is  required 
during  the  engagement  timeframe  and  requires  concise  communication  of  threat 
location  and  characteristics  as  well  as  ownship  and  wingman  data  important  to 
coordination.  Examples  of  possible  symbology  sets  to  augment  SA  are:  target 
locator  lines,  weapon  engagement  zones,  ownship  airspeed  and  altitude,  aircraft 
designators,  and  navigation  aids.  HMD  design  of  the  display  and  symbology 
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should  strive  to  enhance  the  users  understanding  of  the  situation  in  both  phases  of 
flight,  allowing  for  increased  safety,  survivability,  and  lethality. 

Enhance  Weapons  Employment 

Essential  to  survival  in  a  combat  aviation  environment  is  the  pilot’s  ability 
to  defend  against  and  strike  an  enemy  target.  Air-to-air  and  air-to-ground 
engagements  are  the  most  common  mission  scenarios  for  fighter  aircraft.  The 
outcomes  for  these  contests  are  detennined  in  a  matter  of  seconds,  often  by  who 
has  the  ability  to  fire  a  weapon  first.  The  HMD  is  intended  to  provide  the 
advantage  of  being  the  first  to  acquire  and  engage  a  target,  while  maintaining  a 
position  of  advantage.  When  integrated  with  advanced  air-to-air  and  air-to- 
ground  missiles,  the  HMD  advantage  increases  lethality  and  chances  of  survival 
in  a  combat  environment. 

Minimize  Potential  Harm 

Another  aspect  of  increasing  safety  is  to  minimize  the  risk  of  potential 
hann  caused  by  the  system  itself.  Human-Machine  interfaces  must  always  be 
developed  with  the  user  in  mind,  as  a  poor  design  could  inadvertently  cause 
chronic  health  issues  or  fatal  injuries.  Additionally  the  system  must  be  reliable. 

A  failure  of  essential  systems,  such  as  an  interface  that  provides  critical  flight 
information  could  cause  the  pilot  to  lose  control  of  the  aircraft. 

Continuing  further  into  the  solution  space  and  perfonnance  levels,  it  was  important  to 
find  out  information  like,  “What  HMD  attributes  bring  value  to  those  objectives?” 
Provide  Best  Display  Performance 
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This  means  objective  has  to  do  with  how  well  the  technology  performs, 
with  the  purpose  of  supporting  the  enhancement  of  weapons  employment  and 
situation  awareness  objectives.  The  display  must  perfonn  at  a  high  level  so  as  not 
to  induce  negative  physiological  symptoms  that  would  degrade  pilot  SA  and 
tactical  perfonnance,  such  as  nausea,  dizziness,  or  headaches.  Included  in  this 
attribute  will  be  the  functionality  of  the  pertinent  HMD  system  technology 
including;  tracker,  processor,  image  source,  and  display  optics. 

Provide  Best  Human  Factors 

The  physical  relationship  between  the  technology  and  the  user  is  very 
important  to  safety  and  usability  of  the  system.  This  attribute  will  examine  the 
interface  metrics  detennined  to  be  pertinent  to  the  higher  level  objectives  such  as 
pilot  usability,  head-bome  weight,  and  C.G. 

Maximize  Reliability 

An  aircraft  and  its  capabilities  or  only  valuable  if  they  are  available  when 
needed.  A  system  must  be  reliable  enough  so  the  user  is  confident  that  it  will 
function  properly  when  required.  The  probability  that  the  system  will  fail  must  be 
controlled  and  minimized  when  possible.  Intuitively,  the  less  likely  the  system  it 
to  break,  the  less  time  needed  to  repair  it.  This  is  also  important  because  user 
acceptance  is  always  a  metric  of  concern  for  new  technology  or  capabilities  and 
can  often  be  associated  with  the  system’s  reliability. 

Provide  Sustainable  System 

To  accomplish  the  objective  “Maximize  Reliability”  a  sustainable  system 
must  be  required.  Additionally,  as  the  overall  goal  for  HSI  is  to  increase 
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performance  and  decrease  cost  for  entire  life  cycle  of  the  system  it  is  important  to 
develop  metrics  that  will  predict  how  well  a  system  will  meet  this  goal.  The 
sustainment  phase  of  most  programs  is  where  the  majority  of  the  spending  occurs. 
Reducing  this  expenditure  is  a  key  step  in  minimizing  total  life  cycle  cost. 
Minimize  Maintenance  Burden 

The  amount  of  time  necessary  to  perform  an  aircraft  retrofit  and 
maintenance  actions  has  a  direct  effect  on  the  availability  and  reliability  of  the 
system.  A  large  maintenance  burden  is  an  indicator  of  a  system  that  is  more 
likely  to  fail  during  operations  and  therefore  decrease  the  safety  and  operational 
effectiveness  of  the  platfonn. 

A/C  Capability  Integration 

A  unique  attribute  of  modern  helmet  mounted  displays  are  their  ability  to 
provide  additional  SA  by  moving  displays  to  the  pilots  line  of  sight  and  enhancing 
target  engagement  by  cueing  weapons.  Both  of  these  capabilities  are  dependent 
on  how  the  HMD  interfaces  with  aircraft  systems  and  what  capabilities  can  be 
communicated  through  the  processor.  This  is  very  dependent  on  the  aircraft  and 
how  engineers  design  the  avionics  and  weapon  system  management  interfaces. 
Advancement  and  proliferation  of  processor  technology  has  seen  to  it  that  almost 
all  HMD  systems  have  the  ability  to  transfer  any  sort  of  data  required  by  an 
aircraft.  Therefore,  while  important  to  the  fundamental  goals  of  an  HMD,  the 
manner  in  which  the  HMD  is  integrated  into  the  aircraft  will  not  be  an  evaluation 
measure  for  this  effort.  It  is  assumed  that  any  of  the  alternatives  to  be  examined 
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can  meet  the  desired  aircraft  interface  requirements  determined  by  the  program 
office. 


Decomposition  of  these  stakeholder  objectives  and  goals  led  to  an  objectives  relationship 


hierarchy  (Figure  9)  with  tailored  lists  of  system  characteristics  for  the  intended  user- 


mission  environment. 


Figure  9  :  Objectives  Relationship  Hierarchy 

While  this  was  a  very  important  step  to  determine  a  pool  of  required  FIMD 
attributes,  to  be  usable  for  a  decision  analysis  process  the  hierarchy  must  be  made  of 
factors  that  are  measurable,  mutually  exclusive,  and  collectively  exhaustive.  To  meet  the 
mutually  exclusive  constraint,  the  lower  level  objectives  must  not  impact  multiple  high 
level  objectives.  In  other  words,  a  change  in  one  value  factor  cannot  affect  a  change  on 
more  than  one  upper  level  factor.  While  the  current  state  of  the  hierarchy  demonstrates  a 
crossover  of  influence  for  multiple  factors,  to  be  used  for  a  quantitative  decision  analysis, 
the  bottom  levels  of  each  branch  from  the  relationship  hierarchy  were  congregated  to 
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create  a  value  hierarchy  that  retains  the  user-centric  methodology.  A  single  overall 
measure,  that  will  become  the  basis  for  alternative  value  comparison,  was  created  to  sum 
up  the  lower  level  values.  It  is  labeled  as  the  HMD  Value,  Impact  on  Operational 
Effectiveness.  Figure  10  shows  the  final  iteration  that  will  be  the  basis  for  the  remainder 
of  the  DA  process. 


Figure  10  :  Value  Hierarchy 


Step  3  and  4  :  Develop  Evaluation  Measures  and  Create  Value  Functions 

The  lowest  level  of  the  value  hierarchy  shown  above,  in  Step  2,  are  evaluation 
metrics  determined  to  be  appropriate  for  the  framework.  Deriving  and  down-selecting 
the  measures  was  an  iterative  process  that  began  with  a  large  list;  accumulated  based  on 
literature  review,  the  author’s  personal  experience  with  helmet  mounted  displays  and 
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fighter  aircraft,  and  discussions  with  technical  experts  in  the  field.  This  initial  collection 
is  shown  in  its  entirety  in  Appendix  A.  As  is  good  practice  for  the  decision  analysis 
process,  inputs  from  subject  matter  experts,  the  senior  decision  maker,  and  stakeholders 
themselves  were  used  to  remove  those  metrics  that  were  not  under  HMD  vendor  control, 
would  be  fully  defined  in  a  requirements  document,  or  were  deemed  not  important  to  the 
specific  decision  space  because  of  decision  maker  preference  or  technology  capabilities. 

Once  the  objectives  and  associated  values  have  been  elicited  and  refined,  the 
metrics  and  value  function  curves  must  be  developed.  For  this  study,  Steps  3  and  4  are 
presented  together,  describing  each  of  the  evaluation  measures  determined  to  be 
important  to  the  decision  space  and  developing  their  associated  value  function  curves 
based  upon  examination  of  past  research  efforts  and  discussions  with  relevant  HMD 
subject  matter  experts.  Each  value  function  shows  the  metric  on  the  abscissa  and  the 
value  of  the  measure  on  the  ordinate  axis,  0  (least  value)  to  1  (most  value). 

Display  Performance 

Resolution 

As  described  in  Chapter  II,  the  resolution  of  the  display  is  an  indication  for  how 
clear  the  symbology  or  video  will  appear  to  the  user.  For  the  purposes  of  this  research,  a 
pixel  density  measure  will  be  used  to  attain  a  value  number  for  the  resolution  of  an 
alternative.  An  increasing  density  (number  of  pixels  per  degree  of  Field  of  View)  means 
the  image  will  appear  sharper.  A  sharper  display  is  desired  for  operational  use  in  order  to 
delineate  between  symbology  sets  and  avoid  “jagged”  lines  and  curves.  Discussions  with 
Air  Force  Research  Lab  HMD  technical  experts  have  shown  the  requirement  for  display 
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resolution  is  very  dependent  on  what  is  being  displayed.  Sensor  or  synthetic  imagery 
require  much  higher  contrast  levels  than  for  basic  symbology,  so  as  to  allow  the  wearer  to 
delineate  details  in  the  scene.  However,  the  contrast  of  the  sensor  imagery  is  limited  by 
the  sensor  itself  as  the  HMD  cannot  show  more  detail  than  the  sensor  input  is  providing. 
Recognizing  that  a  future  application  of  this  framework  could  include  multiple  curves  for 
evaluation  of  imagery  contrast  as  well  as  basic  symbology,  Figure  1 1  shows  the  value 
function  of  display  resolution  for  symbology  only.  The  developed  curve  indicates  a 
rapidly  increasing  value  as  the  pixel  density  increases  to  about  60  pixels  per  degree  and 
then  trailing  off  to  show  a  lower  rate  of  return  as  the  resolution  increases  beyond  60. 


Figure  11  :  Display  Resolution  Value  Function 

System  Latency 

This  metric  is  a  global  way  of  describing  many  individual  HMD  component 
performances.  The  system  latency,  given  as  a  time  metric,  is  affected  by  the  update  rate 
of  the  tracker  system,  the  speed  of  the  processor,  and  the  refresh  rate  of  the  image  source. 
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One  goal  of  HMD  system  design  is  to  lower  this  time  chain  as  much  as  possible,  until 
there  is  no  noticeable  lag  in  symbology  or  video  as  the  pilot  moves  their  head.  It  is  also 
important  to  note  that  the  apparent  latency  of  an  HMD  is  also  a  function  of  the  aircraft 
systems  that  drive  the  image  or  provide  the  data.  The  speed  at  which  the  sensor  or 
avionics  generates  the  information  and  transfers  it  to  the  HMD  processor  plays  a  role  in 
the  apparent  latency  to  the  pilot.  However,  this  aircraft  based  aspect  is  considered  to 
have  an  effect  on  all  HMD  alternatives  equally  and  so  is  not  taken  into  account  for  this 
framework.  Discussions  with  Air  Force  Research  Lab  HMD  technical  experts  revealed 
there  could  be  multiple  latency  requirements  for  an  HMD  system,  based  upon  its  use  and 
what  is  being  displayed.  A  lag  in  sensor  or  synthetic  imagery  would  be  much  more 
noticeable  and  detrimental  to  a  pilot  than  basic  symbology  and  therefore  would  have 
more  stringent  bounds  and  curves.  To  achieve  simplicity  in  this  initial  framework 
proposal,  the  value  function  (Figure  12)  has  been  formed  based  upon  basic  symbology 
display  only,  recognizing  that  a  future  application  of  the  framework  could  include  a  curve 
for  imagery.  It  shows  the  value  of  the  alternative  will  decrease  as  the  system’s  latency 
increases.  The  steepest  part  of  the  curve,  located  between  2  ms  and  5  ms,  indicates  the 
greatest  rate  of  value  return  for  achieving  a  lower  latency  between  these  points. 
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Figure  12  :  System  Latency  Value  Function 

Contrast 

The  contrast  of  a  visual  medium  is  given  by  the  ratio  between  the  darkest  part  of 
the  display  and  the  lightest  part.  This  characteristic  is  important  to  readability  because  a 
low  contrast  would  make  the  symbology  appear  washed-out  and  difficult  to  delineate. 

For  a  see-through  HMD  it  is  desired  to  be  able  to  view  the  display  in  all  ambient  lighting 
conditions,  and  of  most  concern  would  be  the  high  brightness  scenes  such  as  looking  into 
the  sun  or  viewing  snow  covered  ground  on  a  sunny  day.  While  display  luminance  can 
be  increased  in  an  attempt  to  increase  contrast,  i.e.  making  the  bright  part  sufficiently 
brighter  than  the  dark  parts,  the  transmissivity  of  the  combiner  and  helmet  visor  can  also 
be  decreased  to  decrease  the  apparent  ambient  brightness  reaching  the  user’s  eye. 
However  this  decrease  in  visor  transmissivity  could  have  detrimental  effects  as  the 
overall  scene  will  appear  darker.  Research  has  shown  that,  to  be  readable,  the  contrast  of 
an  image  or  text  should  be  at  least  1.2  (Clarence  E.  Rash,  2000).  Through  discussions 
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with  Air  Force  Research  Lab  HMD  technical  experts,  it  was  found  that  (similarly  to 
latency)  there  could  be  multiple  contrast  value  function  curves.  This  is  because  the 
intended  use  of  the  display  would  have  different  contrast  requirements.  Night  viewing 
contrast  would  be  easier  to  achieve  because  of  the  darker  ambient  scene,  therefore 
changing  the  metric  bounds  to  a  higher  level.  Also,  sensor  video  or  synthetic  imagery 
would  require  a  higher  contrast  for  best  usability  than  basic  symbology  information.  The 
following  value  function  (Figure  13)  has  been  created  to  indicate  the  decision  makers 
preferences  towards  a  day,  symbology  only,  environment;  recognizing  that  the  shape  and 
bounds  could  change  for  other  uses.  The  display  contrast  ratio  for  the  system  used  for 
this  value  function  is  that  which  is  measured  against  a  10,000  fL  ambient  scene,  a 
brightness  commonly  used  in  perfonnance  specifications.  The  curve  shows  a  lower 
bound  at  the  minimum  acceptable  contrast  of  1 .2  and  an  increasing  value  as  the  ratio 
reaches  about  10.  It  then  has  a  gradually  decreasing  slope  to  indicate  a  decreasing  rate  of 
return  for  value. 
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Figure  13  :  Display  Contrast  \ alue  Function 

Field  of  View 

Field  of  View  (FOV),  with  respect  to  a  HMD,  describes  how  much  of  the 
viewer’s  instantaneous  visual  space  can  be  overlaid  with  symbology  or  video.  With 
square  or  rectangular  type  displays,  this  can  be  measured  in  a  vertical  by  horizontal 
manner.  However,  there  are  some  HMDs  that  incorporate  a  circular  or  oval  projection 
medium,  often  measured  in  a  single  diameter  dimension.  Therefore,  this  exercise  will  use 
circular  diameter  measure  as  the  value  metric,  which  can  be  taken  from  a  rectangular  (or 
square)  display  by  inscribing  an  oval  (or  circle)  and  measuring  the  largest  diameter.  As 
the  measure  (degrees)  increases,  the  value  also  increases  in  a  gradually  decreasing  rate  of 
return  manner.  This  shape  is  based  upon  reviewed  literature,  which  finds  a  20  degree 
FOV  is  sufficient  for  easy  tasks,  while  60  degrees  may  be  required  for  more  demanding 
tasks  (M.J  Wells,  1989).  This  trend  can  be  seen  in  Figure  14. 
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Figure  14  :  Field  of  View  Value  Function 


System  Accuracy 

The  accuracy  of  the  system  is  determined  by  the  measurable  deviation  between 
the  symbology  and  the  real  world  object  it  is  intended  to  overlay;  therefore  a  larger 
displacement  means  a  less  accurate  system.  It  is  recognized  that  due  to  tracker  and 
processor  characteristics  the  accuracy  of  HMD  systems  may  vary  through  the  pilot’s  field 
of  regard  (F.O.R),  generally  decreasing  as  distance  from  aircraft  boresight  is  increased. 
Performance  requirements  are  typically  written  for  different  threshold  and  objectives  at 
certain  sections  of  the  F.O.R.  This  study  will  use  the  measured  accuracy  for  the  forward 
facing  section,  about  45  degrees  of  either  side  of  the  aircraft  boresight.  Also,  if 
displaying  a  sensor  fed  video  picture,  the  accuracy  of  the  image  will  be  very  dependent 
on  the  accuracy  of  the  aircraft  sensor  that  is  driving  the  image.  Therefore,  in  an  effort  to 
achieve  simplicity  in  this  initial  analysis,  the  accuracy  of  symbology  only  will  be 
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evaluated  and  not  video  imagery.  If  evaluating  video  or  synthetic  imagery  displayed,  the 
value  function  curve  may  have  a  different  shape.  The  value  function  curve  in  Figure  15 
is  based  upon  research  (Richard  L.  Newman,  1997)  and  discussions  with  Air  Force 
Research  Lab  HMD  technical  experts  who  have  described  a  decreasing  value  as  the 
accuracy  displacement  increases  to  about  5  milliradians.  The  curve  then  levels  out 
towards  0  value  to  indicate  little  to  no  value  of  the  system,  if  less  accurate  than  5 
milliradians. 


Display  Accuracy  displacment  (milliradians),  x5 
Figure  15  :  System  Accuracy  Value  Function 


Human  Factors 

Weight  and  Center  of  Gravity 

In  1991,  the  Air  Force  Research  Lab  conducted  a  study  to  determine  head  and 
neck  criteria  with  respect  to  weight  and  center  of  gravity  limitations  for  ejection  seat 
aircraft.  The  report  entitled  “Interim  Head/Neck  Criterion”  (F.S  Knox,  1991),  plotted  the 
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recommended  boundaries  based  upon  the  entire  head  borne  weight  (helmet  +  mask), 
measured  on  a  Large  ADAM  manikin  head.  A  total  weight  limit  of  5.0  lbs  was 
established  to  insure  survival  during  an  ejection  event  and  limit  pilot  fatigue  and 
perfonnance  detriments.  Center  of  Gravity  limits  were  given  in  reference  to  the  X,  Y, 
and  Z  axes  as  illustrated  in  Figure  16.  These  boundaries  were  -0.8  to  0.5  inches  (x-axis), 
±0.15  inches  (y-axis),  and  0.5  to  1.5  inches  (z-axis).  The  space  enclosed  within  these 
coordinal  limitations  is  known  as  the  “Knox  Box”. 
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Figure  16  :  Center  of  Gravity  Axes  for  the  Head 

Figure  17  and  18  are  the  Total  Weight  and  C.G  deviation  value  functions  based  upon  the 
“Knox  Box”  boundaries.  The  weight  value  function  shows  an  increase  in  value  as  the 
weight  is  decreased  from  the  “Knox  Box”  requirement  of  5  lbs,  gradually  decreasing  in 
rate  of  value  return  as  the  weight  is  decreased. 
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Figure  17  :  Headborne  Weight  Value  Function 


Figure  18  :  Center  of  Gravity  Value  Function 
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The  center  of  gravity  value  functions  are  broken  out  into  the  3  axes  to  align  with 
the  “Knox  Box”  requirements.  Each  of  the  curves  indicates  the  highest  value  is  gained 
when  the  center  of  gravity  deviation  is  zero,  or  the  same  as  the  C.G.  of  the  head.  The  x- 
axis  curve  reflects  the  different  measure  requirements  of  the  “Knox  Box,  aft  to  forward  of 
head  C.G.,  with  the  more  value  being  gained  the  closer  the  HMD  C.G.  is  moved  toward 
that  point.  In  order  to  calculate  a  single  value  measure  for  the  center  of  gravity  metric,  the 
additive  method  will  be  applied  using  equal  weights  for  each  axis  value  function. 

Pilot  Satisfaction  (Usability) 

As  expressed  in  Chapter  II,  user  satisfaction  with  the  system  is  an  extremely 
important  metric  for  a  successful  program.  An  interface  that  is  not  user  friendly  or 
impairs  any  critical  tasks  is  not  acceptable.  Yet,  this  is  one  of  the  most  difficult 
characteristics  to  grade  in  a  quantitative,  repeatable  manner  as  it  is  very  subject  to  user 
bias  and  personality.  Utilized  commonly  in  the  human  factors  engineering  community, 
pilot  feedback  questionnaires  will  be  the  basis  for  scoring  this  metric.  Questions  will  be 
posed  to  elicit  approval  scores  on  important  aspects  of  the  interface  such  as;  Ability  to 
don/ doff  the  system,  interactions  with  canopy  and  other  obstructions,  effect  on  external 
field  of  view,  fit  and  comfort,  etc.  Example  questionnaire  can  be  found  in  Appendix  B. 
The  scores  for  each  question  are  added  together  to  create  a  single  Pilot  Satisfaction  score, 
maximum  of  100.  This  score  is  used  to  create  the  value  function  seen  in  Figure  19. 
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Figure  19  :  Pilot  Satisfaction  Value  Function 

Viewing  Eyebox  ( Exit  Pupil) 

An  optic’s  Viewing  Eyebox  is  the  area  in  which  the  eye  can  be  placed  and  still  see 
the  image.  Depicted  in  Figure  20  ,  the  user  could  lose  some  or  the  entire  image  if  the 
FIMD  shifts  to  the  point  that  the  eye  is  no  longer  within  the  viewing  eyebox.  There  are 
two  optical  design  approaches  common  in  HMDs,  pupil-fonning  and  non-pupil-forming. 
While  not  necessary  to  get  into  the  details  of  each  design,  it  is  important  to  note  that  in 
pupil-forming  optics,  the  viewing  eyebox  is  also  known  as  the  exit  pupil  and  there  will  be 
no  image  available  directly  outside  the  exit  pupil.  And  in  a  non-pupil-fonning  design 
vignetting  or  clipping  of  symbology  will  be  seen  as  the  viewer  moves  outside  the  eyebox. 
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Non-Pupil-Forming  Optics 


Figure  20  :  HMD  Optic  Designs 

No  matter  the  optic  choice  of  the  developer,  it  is  important  to  have  a  sufficiently  large 
viewing  eyebox  so  as  to  insure  the  display  is  still  useable  within  an  expected  amount  of 
helmet  shift.  Research  has  shown  a  minimum  of  12  mm  is  acceptable,  though  the 
operationally  successful  IHADSS  HMD  has  just  a  10  mm  exit  pupil  (Clarence  E.  Rash, 
2000).  For  this  effort  a  minimum  boundary  of  10  mm  was  used  and  the  shape  of  the 
curve  was  created  based  upon  discussions  with  AFRL  technical  experts  to  show  a  rapidly 
increasing  value  as  the  eye  box  size  is  increased  to  about  20  mm,  followed  by  a  gradually 
decreasing  slope  to  indicate  a  decreasing  rate  of  value  return.  The  function  is  shown  in 
Figure  2 1 . 
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Figure  21  :  Viewing  Eyebox  Value  Function 

Provide  Sustainable  System 

HMD  Mean  Time  Between  Maintenance  Event  (MTBME) 

One  metric  for  the  HMD  maintainability  and  thus  sustainability  is  the  time 
between  the  scheduled  and  unscheduled  maintenance  events  that  require  repair  of  the 
equipment  on  or  off  the  aircraft.  These  events  consist  of  preventative  or  corrective 
maintenance  actions  required  to  retain  or  restore  system  hardware  in  a  functional  state. 
This  mean  time  between  maintenance  events  (MTBE)  should  be  maximized  to  ensure 
continued  availability  of  the  system.  The  bounds  of  this  measure  are  very  dependent  on 
the  aircraft  platform  and  the  decision  maker’s  preferences.  Elicitation  of  the  example 
senior  decision  maker’s  preferences  has  determined  a  lower  and  upper  bound  of  800  and 
1200  hours  respectively,  with  a  linear  (unchanging  rate  of  return)  between  the  bounds. 
Figure  22,  shows  a  value  function  curve  for  MTBME  derived  from  the  example  SDM. 
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Figure  22  :  MTBME  Value  Function 

100%  Warranty  Period 

Another  indicator  of  the  expected  sustainability  of  the  system  is  the  amount  of  full 
coverage  warranty  the  seller  agrees  to  apply.  As  the  contractor’s  primary  goal  is  to  make 
money,  they’re  warranty  period  should  be  directly  related  to  how  long  they  expect  the 
system  to  function  well.  A  longer  warranty  offer  could  indicate  less  likelihood  for 
maintenance  action  during  that  time  period,  and  therefore  better  sustainability.  This 
increasing  linear  trend  for  number  of  warranty  years  and  value  is  shown  in  Figure  23. 
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Figure  23  :  Warranty  Period  \ alue  Function 


Growth  Capability 

As  aircraft  platforms  are  modernized  and  upgraded,  software  mods  will  require 
extra  amounts  of  memory  and  processing  capacity.  This  is  an  expected  and  understood 
requirement  and  an  alternative’s  capability  to  account  for  this  should  bring  extra  value.  It 
was  found  that  the  important  metrics  for  determine  ability  to  accommodate  system 
growth  were  CPU  process  utilization,  memory  capacity,  and  disk  space/storage.  To 
synthesize  a  single  value  measure  from  these  three  metrics,  a  similar  technique  as  the 
center  of  gravity  metrics  was  used.  Each  of  the  respective  growth  metrics  were  given 
their  individual  value  functions  and  then  a  direct  weight  (equal  weights),  additive  method 
was  used  to  calculate  a  Growth  Capability  value.  The  curves  are  based  upon  the  percent 
of  the  total  metric  that  is  unused  in  the  delivered  system,  i.e.  the  amount  available  for 
future  upgrades.  Intuitively,  the  higher  available  percentage,  the  more  value  is  gained. 
Figures  24  shows  the  three  value  function  curves  for  the  individual  metrics. 
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Figure  24  :  Growth  Capability  \ alue  Function 


Maintainer  Satisfaction 

This  metric  is  very  similar  to  the  Pilot  Satisfaction  score  described  earlier  in  this 
section,  with  the  exception  of  being  taken  from  the  maintainer’s  point  of  view.  The 
questionnaire  was  focused  on  the  maintainer’s  ability  to  dismantle  the  necessary 
components,  ease  of  daily  maintenance  tasks,  preflight  checks  of  the  system,  etc. 
(questionnaire  can  be  found  in  Appendix  B).  These  same  scoring  method  and  additive 
weighting  functions  were  used  to  create  the  Maintainer  Satisfaction  score.  Again,  this 
score  is  plotted  against  value  to  create  the  value  function  found  in  Figure  25. 
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Figure  25  :  Maintainer  Satisfaction  Value  Function 

Minimize  Maintenance  Burden 

A/C  Downtime  for  Retrofit 

The  amount  of  time  the  aircraft  is  unavailable  during  the  hardware  retrofit  process 
is  a  metric  that  can  be  used  to  determine  system  complexity  and  maintenance  burden.  It 
is  reasonable  to  associate  a  lower  retrofit  downtime  with  a  less  complex  system.  The 
boundaries  of  what  is  acceptable  would  be  determined  by  the  senior  decision  maker  for 
the  platform  under  consideration.  In  Figure  26,  the  value  function  for  retrofit  downtime 
is  shown  with  boundaries  (10-40  hours)  suggested  from  the  example  decision  maker. 
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Figure  26  :  Retrofit  Downtime  Value  Function 

Retrofit  Location 

The  location  at  which  the  retrofit  must  be  performed  indicates  the  level  of 
complexity  of  the  process  and  therefore  can  have  an  effect  on  the  future  reliability.  The 
simplest  retrofits  can  be  performed  in  the  field,  i.e.  at  operational  bases,  by  DoD 
employees.  For  more  involved  modifications,  the  effort  would  have  to  be  accomplished 
at  a  Depot  location  with  more  availability  of  expertise  and  tooling.  If  the  contractor 
detennines  that  the  modification  is  very  complicated  or  doesn’t  want  to  accept  the  risk  of 
someone  else  performing  the  retrofit,  it  can  often  be  done  by  their  own  personnel, 
sometime  at  their  own  location.  As  system  complexity  is  an  aspect  that  the  decision 
maker  would  be  interested  in  minimizing  for  the  purposes  of  maximizing  reliability,  a 
decreasing  categorical  value  function  can  be  used  since  there  is  no  intennediate  gradation 
between  location  types  (Figure  27). 
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Figure  27  :  Retrofit  Location  Value  Function 

Power  Required 

This  is  an  evaluation  metric  whose  boundaries  and  importance  are  very  dependent 
on  the  application  and  specific  customer  platform.  Aircraft  must  share  power  supply  with 
many  systems  at  once  and  thus  allowance  to  new  systems  is  carefully  controlled  so  as  not 
to  cause  detriment  to  other  more  critical  components.  Generally  the  maximum  allowable 
power  consumption  is  a  requirement  levied  on  all  alternatives.  This  value  function 
(Figure  28)  shows  a  decreasing  value  as  the  percentage  of  that  maximum  is  increase.  In 
other  words,  if  an  alternative  only  requires  25%  of  the  allowable  requirement  it  will  gain 
more  valuable  compared  to  another  that  needs  75%  of  the  maximum. 
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Figure  28  :  Power  Required  Value  Function 

Maximum  Repair  RT  (MaxRT)  for  95%  of  Off-Equipment  Repairs 

Off-equipment  maintenance  is  considered  to  be  those  actions  that  require  removal 
of  HMD  components  from  the  aircraft  to  be  fixed  in  the  maintenance  shop.  If  parts  of  the 
system  are  removed  from  the  platfonn,  then  the  particular  aircraft  has  lost  functionality  of 
the  HMD.  Typically  spare  parts  can  be  used  to  immediately  restore  operation,  but  then 
the  spare  pool  is  also  being  impacted.  The  amount  of  time  expected  to  return  those  parts 
to  full  working  order  is  an  indication  of  the  maintenance  burden  for  the  total  system. 

This  evaluation  measure  is  the  amount  of  time  95%  percent  of  all  off-equipment  repairs 
can  be  completed.  Decreasing  this  amount  of  time  adds  value  to  the  system,  as  indicated 
in  Figure  29. 


85 


Hours,  x17 

Figure  29  :  MaxRT for  95%  Value  Function 

Step  5  :  Weight  the  Objectives  Hierarchy 

While  the  value  hierarchy  offers  a  thorough  understanding  of  the  important 
evaluation  measures  to  the  decision  space,  it  must  be  recognized  that  not  all  the  measures 
are  of  equal  importance  to  the  decision.  Therefore  a  weighting  schema  must  be 
employed  to  account  for  the  decision  maker’s  preferences  between  the  factors.  A  bottom 
up  approach  was  used,  in  that  the  local  weights  of  the  lowest  level  objective  values  were 
found  using  the  direct  weighting  method  and  repeated  for  each  higher  level  of  the 
hierarchy.  An  example  of  the  elicitation  process  is  shown  in  Figure  30  for  one  of  the 
lowest  branches  of  the  value  hierarchy,  where  X  represents  the  metric  with  the  lowest 
amount  of  weight  indicated  by  the  decision  maker  and  the  multipliers  represent  the  DM’s 
answer  to  the  question  “By  what  multiplying  factor  do  you  believe  this  one  is  more 
important  than  the  lowest?”.  For  example,  “Pilot  Satisfaction  is  five  times  more 
important  than  the  Weight”. 


86 


x  + 1 .5*  +  5.y  +  5x  =  1 

Solve  for  x 

*  ~  ^  Plug  in  x  value 


0.10 

0.14 

0.48 

0.29 


Figure  30  :  Example  of  Weight  Elicitation  Process 

The  following  Table  shows  the  local  weighting  results  of  the  example  senior  decision 
maker  preference  elicitation  process. 


Table  4  :  Calculated  Local  Weights 


Display  Performance 

0.33 

Resolution 

0.22 

System  Latency 

0.22 

Contrast 

0.22 

Field  Of  View 

0.11 

System  Accuracy 

0.22 

Sustainable  System 

0.17 

HMD  MTBME 

0.33 

Warranty  Period 

0.17 

Growth  Capability 

0.08 

Maintainer  Satisfaction 

0.42 

Human  Factors 

0.42 

Weight 

0.10 

Center  of  Gravity 

0.14 

Pilot  Satisfaction 

0.48 

Viewing  Eyebox 

0.29 

Maintenance  Burden 

0.08 

A/C  Downtime  for  RF 

0.13 

RF  Location 

0.13 

Power  Required 

0.25 

MaxRT for  95% 

0.5 
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Figure  3 1  below,  illustrates  the  value  hierarchy  with  the  associated  calculated  global 
weights,  or  impact  on  total  HMD  value. 


Figure  31  :  Value  Hierarchy  with  Global  Weights  (rounded  to  the  hundredth) 


Step  6  :  Alternative  Generation 

This  is  the  step  where  the  alternatives  and  technical  details  are  collected.  The  two 
alternatives  to  be  evaluated  in  this  framework  are  the  Scorpion  Helmet  Mounted  Cueing 
System  and  the  Joint  Helmet  Mounted  Cueing  System  (JHMCS).  These  alternatives 
were  chosen  because  they  have  been  fielded  and  used  on  U.S.  Air  Force  fighter  aircraft 
for  years  and  should  have  readily  available  technical  information.  Ideally,  alternatives 
would  include  those  solution  that  may  still  be  in  development  or  test  along  with  legacy 
hardware. 
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Scorpion  Helmet  Mounted  Cueing  System 


Figure  32  :  Alternative  1-Scorpion  (Thales  Visionix  Technical  Overview) 

The  Scorpion  Helmet  Mounted  Cueing  System,  depicted  above  in  Figure  32,  was 
developed  by  Gentex  Visionix,  a  division  of  Gentex  Corporation  that  is  now  a  subsidiary 
of  Thales  Group.  Designed  in  the  mid  2000s,  this  HMD  features  LCD  waveguide 
technology  to  present  full  color  symbology  or  video  imagery  on  a  monocular  display 
module  positioned  in  front  of  the  pilot’s  eye.  This  display  position  is  adjustable  to 
accommodate  differing  pilot  sizes  and  additional  head  worn  protection  devices  such  as 
laser  eye  protection  spectacles,  which  can  be  worn  between  the  monocle  and  pilot’s  eyes. 
Originally,  the  Scorpion  system  incorporated  a  magnetic  type  tracker  system  only,  but 
has  expanded  to  include  a  hybrid-inertial  tracker  option.  The  Hybrid  Optical  based 
Inertial  Tracker  (HObIT)  technology  uses  an  inertial  sensor  to  detennine  head  orientation 
and  is  recalibrated  by  the  integrated  optical  sensor.  The  HObIT  system  employs  fiducial 
stickers  mounted  to  the  aircraft  canopy  in  a  constellation  pattern  above  the  pilot’s  head. 
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These  stickers  each  have  bar  code  designs  printed  on  them,  and  relay  unique  light 
patterns  back  to  the  optical  sensor  on  the  helmet,  creating  a  triangulation  type  calibration. 
Scorpion  has  a  nighttime  color  cueing  capability,  as  the  banana  bar  that  supports  the 
display  and  tracker  components  also  incorporates  the  same  NVG  mount  currently  in  use. 
Legacy  goggles  can  be  attached  and  are  deployed  immediately  forward  of  the  monocle, 
so  the  pilot  would  look  through  the  display  then  the  NVGs. 

Joint  Helmet  Mounted  Cueing  System  (JHMCS) 


Figure  33  :  Alternative  2  -  JHMCS 


The  Joint  Helmet  Mounted  Cueing  System  (JHMCS),  pictured  above  in  Figure 
33,  is  a  HMD  system  produced  by  Vision  Systems  International  (VSI)  as  part  of  a  joint 
development  and  acquisition  program  between  the  U.S.  Air  Force  and  Navy.  The  system 
incorporates  a  magnetic  type  tracker  and  aircraft  mounted  processor  unit.  The  CRT 
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image  source  projects  monochrome  symbology,  synthetic,  or  sensor  pod  imagery  directly 
onto  the  pilot’s  visor  and  provides  a  20  degree  circular  F.O.V.  Night  vision  cueing 
capability  is  offered  with  night  module  options  such  as,  the  Night  Visions  Cueing  Display 
(NVCD)  which  consists  of  a  set  of  modified  legacy  night  vision  goggles  that  attaches  a 
small  display  generator  on  the  end  of  the  NVG  tube. 

Step  7  :  Alternative  Scoring 

To  score  each  alternative,  system  specifications  were  examined  and  questions 
were  inquired  of  the  respective  manufacturers  to  gain  the  necessary  information.  In  a  real 
world  application  of  this  model,  it  would  be  ideal  to  have  the  Pilot  Satisfaction  score 
come  from  an  operator  who  has  used  all  the  HMD  alternatives  on  the  platform  under 
consideration.  This  would  assure  answers  on  the  questionnaire  were  tailored  to  the 
aircraft-mission  combination.  However,  as  this  research  was  an  academic  effort  and  not 
an  actual  alternative  examination,  it  was  not  possible  to  gain  these  scores  from  a  single 
pilot  on  all  the  alternatives.  Therefore,  pilots  familiar  with  the  alternatives  themselves 
were  sought  out  and  their  questionnaire  scores  were  used.  This  will  be  a  subject  of  note 
during  the  Conclusions  and  Recommendations  section  of  the  research.  The  metrics  were 
input  to  an  Excel  based  tool,  which  then  calculated  the  value  scores  and  performed  the 
additive  weighting  function. 

Summary 

This  chapter  has  provided  a  detailed  review  of  the  first  seven  steps  of  the  Value- 
Focused  Thinking  methodology,  applied  to  the  ‘ lack  of  HMD  ’  problem  space.  After 
defining  the  problem  and  determining  what  objectives  would  bring  value  to  the  solution, 
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evaluation  measures  and  associated  value  functions  were  developed.  Continuing  down 
the  left  side  of  the  VFT  process  “V”,  preference  weights  were  elicited  from  the  decision 
maker  using  the  direct  weighting  method.  The  alternative  generation  step  produced  three 
possible  solutions  to  examine  and  grade  in  the  alternative  scoring  process.  The 
Detenninistic  Analysis  and  Sensitivity  Analysis  steps  will  be  presented  in  Chapter  4. 
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IV.  Analysis  and  Results 


Chapter  Overview 

Chapter  IV  works  up  the  right  side  of  the  VFT  process  “V”  to  analyze  the  results 
of  the  model  inputs.  A  deterministic  analysis  will  be  perfonned  to  detennine  the 
suggested  solution  for  the  decision  maker.  This  will  be  followed  by  a  sensitivity  analysis 
that  will  help  verify  the  result  of  the  detenninistic  analysis  and  will  provide  more  insight 
to  the  decision  maker  about  alternative  dominance  when  subjected  to  changes  in  model 
parameters. 

Step  8  :  Deterministic  Analysis 

To  calculate  a  global  value  score,  ‘ HMD  Value,  Impact  on  O.E’,  the  metrics, 
value  curves,  and  weights  derived  in  the  previous  steps  were  populated  into  a 
spreadsheet,  shown  below  in  Figure  34. 


Figure  34  :  Hierarchy  used  to  determine  HMD  value 
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Using  the  direct  weighting  and  additive  methods,  a  value  score  for  each  alternative  was 
output  generated,  shown  in  Table  5. 

Table  5  :  Weighted  Value  Scores 


HMD  Value,  Impact  on 
Operational  Effectiveness 

Maximum  Value 

1.000 

Alternative  1 

0.481 

Alternative  2 

0.436 

A  graphical  depiction  of  the  global  contribution  each  evaluation  metric  made  to  the  total 
score  is  shown  in  Figure  35.  The  percentage  value  next  to  the  metrics  labels  indicates  the 
global  weight  of  each  evaluation  measure.  It  can  be  seen  that  the  resultant  scores  indicate 
Alternative  1  holds  the  most  value  for  this  decision  situation.  While  Alternative  1  shows 
lower  resolution,  the  better  latency  and  center  of  gravity,  make  it  the  more  valued 
alternative. 

However,  Alternative  2  sees  only  a  9.3%  difference  from  Alternative  1.  This  can 
be  attributed  to  the  equalization  required,  due  to  lack  of  complete  operational  test  data. 
For  any  missing  data,  because  of  releasability  or  access  to  infonnation,  the  figures  values 
came  from  the  published  HMD  specifications.  Not  only  did  this  technique  result  in 
similar  values  for  each  alternative,  but  for  a  few  metrics,  the  specification  value  was  the 
lower  bound  of  the  value  function.  It  is  expected,  when  populated  with  a  full  suite  of 
actual  test  data,  the  framework  would  produce  a  resultant  set  of  scores  that  would  have 
more  delineation  between  alternatives. 
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Figure  35:  Contributions  of  Evaluation  Metrics 

Figure  36,  shows  another  illustration  of  the  contributing  factors  that  account  for 
the  score  differential  between  the  two  alternatives.  This  waterfall  diagram  contains  the 
total  value  for  each  alternative,  the  green  bars,  and  the  evaluation  measure  deltas  that 
created  the  total  differential,  in  the  blue  and  red  bars.  All  other  measures  were  of  equal 
value.  In  this  fonnat  it  can  be  seen  that  Alternative  1  outscored  or  was  equal  in  value  for 
all  measure  except  the  three  colored  in  red  (resolution,  weight,  and  viewing  eyebox). 
These  three  scores  accounted  for  23.3%  of  the  global  weight,  which  means  Alternative  1 
holds  as  much  or  more  value  for  over  three  quarters  of  all  the  evaluation  measures. 
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Alternative  2 : 0.436 
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Figure  36  :  Waterfall  Diagram 

Step  9  :  Sensitivity  Analysis 

A  sensitivity  analysis  is  used  to  examine  the  global  impact  of  changes  to  model 
inputs.  For  this  study,  a  rank  order  sensitivity  analysis  was  applied,  which  explored  the 
possibility  of  a  reverse  in  alternative’s  rank  based  upon  a  modification  to  the  weighting 
schema.  In  Figure  37,  the  abscissa  represents  the  preference  weight  of  the  Provide  Best 
Human  Factors  objective,  varying  from  0  to  1  and  proportionately  adjusting  the 
remaining  three  objective  weights  on  the  same  level.  The  ordinate  is  the  global  value  of 
the  alternatives,  which  are  represented  as  the  red  and  blue  trend  lines.  The  green  vertical 
line  indicates  the  decision  maker’s  original  preference  for  this  objective  (0.42)  and  the 
intersection  points  correspond  with  the  initial  values  found  previously  in  Step  8. 
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Figure  37  :  Sensitivity  Graph 

Upon  examination  of  the  plot,  it  can  be  seen  that  Alternative  1  maintains  a  higher 
value  until  the  weight  is  varied  to  equal  0.72,  at  which  point  Alternative  2  becomes  the 
choice  that  brings  the  decision  space  the  most  value.  In  other  words,  if  the  decision 
maker  were  to  reweight  this  objective  higher  than  0.72,  the  dominant  alternative  would 
change.  However,  this  is  a  significant  7 1 .4%  change  to  the  original  assessed  weight.  The 
sensitivity  to  changes  in  the  other  three  value  hierarchy  objective  weights  had  even  less 
of  an  effect  on  rank  order,  requiring  a  larger  change  from  the  original  preference  weights. 
Display  Perfonnance  requires  decrease  by  91.5%,  Sustainability  never  switches 
dominance,  and  Maintenance  Burden  requires  increase  by  92%. 
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Step  10  :  Conclusions  and  Recommendations 

This  step  should  present  the  analytical  findings  to  the  decision  maker  and  provide 
insight  that  will  strive  for  clarity  of  action.  While  not  incorporated  into  the  framework 
itself,  cost  is  an  aspect  of  the  decision  space  that  cannot  be  overlooked.  Based  on 
literature  review  it  was  determined  that  presenting  cost  against  value  after  the  initial 
analysis  would  be  the  most  efficient  way  to  include  the  monetary  factor.  Figure  38  shows 
the  two  alternatives  plotted  on  a  Value  vs.  Cost  graph.  The  cost  is  the  estimated  expense 
for  a  single  HMD  kit  (unit  cost  or  kit  cost)  and  the  value  is  the  quantity  calculated  in  Step 


8,  during  the  deterministic  analysis. 
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Figure  38  :  HMD  Value  vs.  Cost 
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Figure  38  clearly  indicates  Alternative  1  has  the  slightly  higher  value  and  less  cost,  both 
are  desired  attributes  for  a  solution.  If  this  was  the  only  relevant  data,  the  analyst  would 
recommend  Alternative  1  be  selected  for  integration  on  the  decision  maker’s  aircraft. 
Naturally  other  cost  factors  could  be  included  such  as  estimated  operations  and  support 
(O&S)  cost  or  total  lifecycle  cost  (LCC). 

Summary 

This  chapter  has  provided  an  analysis  of  the  results  gained  from  the  decision 
model  created  in  Chapter  III.  An  initial  ranking  was  detennined  based  upon  original 
decision  maker  preferences  and  HMD  metrics,  detennining  that  for  this  particular  set  of 
alternatives  and  senior  decision  maker,  Alternative  X  was  deemed  to  provide  more  value. 
The  individual  contributions  of  each  evaluation  metric  on  the  global  score  were 
examined.  A  sensitivity  analysis  was  performed  on  the  model  to  examine  any 
possibilities  of  rank  order  switching  based  upon  a  change  in  weighting  preference  or 
evaluation  measure. 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

The  purpose  of  this  research  was  to  develop  and  present  an  evaluation  framework 
that  could  provide  the  basis  for  a  tool  usable  by  program  office  decision  makers.  The 
framework  was  to  incorporate  Human  Systems  Integration  principles  in  order  to  assist  in 
the  identification  of  a  COTS  HMD  solution  that  would  best  optimize  performance  and 
minimize  total  life  cycle  costs  of  the  system.  The  VFT  decision  analysis  method  was 
applied  to  a  theoretical  decision  context  of  a  fighter  platfonn  needing  an  HMD  system. 
Proceeding  through  the  process,  objectives  relevant  to  the  context  were  elicited  and 
evaluation  measures  developed.  Existing  HMD  technology  alternatives  with  readily 
available  infonnation  were  examined  and  scored  within  the  framework. 

This  chapter  will  discuss  the  significance  of  this  research,  primarily  its  possible 
implications  on  assisting  the  DoD  acquisition  community  during  the  analysis  of 
alternatives  phase  of  the  product  life  cycle.  Recognizing  limitations  in  resources  and 
scope,  recommendations  will  be  made  for  future  follow-on  actions  and  research  efforts  in 
order  to  increase  the  effectiveness  of  the  demonstrated  framework. 

Investigative  Questions  Answered 

What  are  the  major  objectives  for  HMD  use  in  a  fighter  aircraft? 

The  objectives  elicited  and  derived  in  Step  2  of  this  framework  were  based  upon 
literature  review,  discussions  with  members  of  the  user  community  and  technical  experts, 
and  the  author’s  experience  with  HMD  technology.  The  top  down  approach  used  to 
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define  the  objectives  was  based  upon  the  principles  of  Value-Focused  Thinking  decision 
analysis,  uniquely  applied  to  the  operational  combat  aircraft  environment. 

What  HMD  factors/attributes,  with  an  emphasis  on  HSI principles,  are  important  when 
comparing  technology  solutions? 

Step  3,  Develop  Evaluation  Measures,  facilitated  the  collection  of  an  exhaustive 
list  of  HMD  attributes.  This  compilation  was  paired  down  to  accommodate  analysis  by 
examining  each  factor  for  their  contribution  to  the  decision  space  and  relevance  to 
modem  technology  capabilities.  Those  that  were  deemed  to  be  sufficiently  defined  in  a 
requirements  documents  were  also  removed  as  a  continued  gradation  after  meeting  the 
minimum  was  not  value  added  to  the  decision.  The  final  list  of  17  metrics  was  used 
through  the  rest  of  the  assessment  and  analysis  process. 

How  should  the  factors  be  weighted? 

As  is  required  in  the  VFT  methodology,  decision  maker’s  preference  was 
represented  by  developing  a  weighting  schema  for  the  factors.  This  process  is  key  to 
tailoring  the  results  for  the  unique  decision  space.  In  a  case  such  as  this,  weighting  can 
be  used  to  adapt  the  framework  for  individual  aircraft  program  offices,  military 
environments,  and  decision  maker  personalities. 

What  are  the  value  metrics  for  those  factors? 

Values  were  calculated  using  value  function  curves  that  detennined  a  score  based 
on  an  evaluation  measure  input.  The  shape  and  bounds  of  the  curves  were  created  based 
upon  human  performance  characteristics  and  decision  maker  preferences,  which  shows 
another  way  this  framework  can  be  tailored  for  use  in  a  variety  of  manners. 
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How  sensitive  is  the  preferred  solution  to  framework  parameters? 

One  sensitivity  analysis  was  performed  on  this  framework,  which  showed  a  rank 
order  sensitivity  to  changes  in  weight  of  a  top  level  objective.  It  is  expected  that  with 
further  analysis  more  value  sensitivity  could  be  found. 

Significance  of  Research 

Though  directed  by  military  handbooks  and  instructions,  the  incorporation  of 
human  systems  integration  (HSI)  principles  is  still  a  struggle  in  DoD  acquisition 
programs.  While  they  are  finding  a  foothold  during  requirements  writing,  no  quantitative 
analysis  of  COTS  equipment  against  the  HSI  domains  has  been  applied  in  a  mainstream 
forum.  Furthermore,  selection  of  COTS  solutions  is  often  performed  in  a  manner  that 
could  miss  influences  on  fundamental  objectives,  which  has  been  shown  to  have  an 
impact  later  in  the  system’s  life  cycle.  The  evaluation  framework  presented  in  this  thesis 
incorporates  many  HSI  principles  in  a  repeatable,  objective  manner.  Tailorable  to 
specific  platforms  and  needs  of  the  user,  a  model  of  this  nature  could  be  used  to  assist  in 
consistent  and  justified  decision  making  for  a  variety  of  programs. 

Recommendations  for  Action 

The  following  are  recommendations  for  actions  that  will  immediately  bring  added 
utility  to  the  framework  presented  in  this  study. 

Apply  Weighting  to  Questionnaires 

To  improve  the  accuracy  of  the  pilot  and  maintainer  questionnaires  and  value 
metric,  each  individual  prompt  can  be  weighted.  This  weight  would  be  assigned  by  the 
subject  completing  the  feedback  form  and  would  represent  their  preference  on  specific 
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characteristics  on  the  examined  HMD  system.  Similar  to  weighting  the  hierarchy,  this 
would  add  additional  aircraft  specificity,  as  pilots  and  maintainers  from  different 
platforms  may  have  different  views  on  what  is  important  based  upon  their  environment 
and  mission. 

Additional  Value  Function  Curves 

As  mentioned  in  Step  4,  Create  Value  Function  Curves,  some  evaluation  metrics 
such  as  display  resolution,  contrast,  and  latency  can  have  completely  different  value 
curves  based  upon  what  is  being  displayed  or  the  intended  environment  (Day  vs.  Night 
usage).  To  enhance  the  completeness  and  ensure  a  robust  decision  framework,  these 
additional  value  functions  can  be  integrated  into  the  model.  Supplementary  literature 
reviews  and  input  from  technical  experts  would  occur  to  ensure  accurate  curves  that 
account  for  physiological  capabilities  and  decision  maker’s  viewpoint  on  the  attributes. 

Evaluate  More  Alternatives 

Due  to  time  and  resource  constraints,  this  effort  could  only  apply  the  evaluation 
framework  on  two  example  alternatives,  with  some  limitations  on  metric  availability. 
While  this  was  enough  to  demonstrate  feasibility  and  usability  of  the  model,  it  is  desired 
to  gather  infonnation  and  metrics  on  additional  HMD  systems  for  examination  and 
comparison.  Working  with  an  aircraft  System  Program  Office  in  need  for  this  type  of 
evaluation  would  also  add  to  the  validity  of  the  framework,  as  complete  tailoring  could 
be  perfonned  for  the  needs  of  the  customer. 
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Apply  Life  Cycle  Cost  vs.  Value  Comparison 

Step  10  of  this  research  presented  a  basic  way  to  communicate  two  important 
metrics  in  the  acquisition  community,  Perfonnance  (value)  and  Cost.  However,  the  cost 
figure  available  at  the  time  of  authorship  was  the  cost  per  unit  of  the  HMD  systems. 

While  this  is  important  to  the  decision  space,  a  more  desirable  data  point  would  be  the 
estimated  life  cycle  cost  of  owning  the  system.  As  the  goal  of  HSI  is  to  maximize 
perfonnance  while  minimizing  total  life  cycle  cost,  a  graph  that  presents  both  of  these 
metrics  on  one  medium  would  be  extremely  valuable  to  the  decision  maker. 

Recommendations  for  Future  Research 

The  following  are  suggestions  for  future  research  to  enhance  framework  fidelity 
and  application. 

Incorporate  Additional  Acquisition  Objectives 

Discussions  with  Senior  Decision  makers  and  other  acquisition  professionals  have 
clarified  that  in  reality  there  are  4  overarching  objectives  for  an  acquisition  program  that 
must  be  carefully  examined  and  balanced;  Cost,  Schedule,  Performance  and  Risk. 

During  an  analysis  of  HMD  alternatives,  these  objectives  would  all  play  a  part  in  which 
solution  is  chosen.  As  illustrated  in  Figure  39,  the  evaluation  framework  presented  in  this 
thesis  effort  has  encompassed  the  Performance  aspects  of  the  system,  and  the  value  an 
alternative  brings  to  that  objective.  To  maximize  the  framework’s  utility  in  the  military 
acquisition  community,  it  would  be  ideal  to  integrate  the  other  three  objectives  as  part  of 
an  analysis.  For  example,  Performance  or  the  HMD  Value,  Impact  on  Operational 
Effectiveness  metric  could  be  plotted  against  Cost  and  also  Schedule  so  as  to  allow  for 
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comparison  of  often  competing  objectives.  Risk  could  possibly  be  incorporated  as  a 
multiplier  of  the  value  score,  i.e.  a  high  risk  alternative  would  have  a  risk  multiplier  less 
than  one  which  would  then  lower  the  value  score  when  multiplied  on  HMD  Value, 

Impact  on  Operation  Effectiveness.  Future  research  efforts  could  determine  the  optimum 
way  to  integrate  these  acquisition  objectives. 


Figure  39  :  Acquisition  Objective  Hierarchy 


Integrate  Additional  HSI  Domains 

As  described  in  Chapter  II,  Human  Systems  Integration  (HSI)  is  broken  out  into  9 
domains  that  when  holistically  applied  can  increase  performance  and  decrease  total  life 
cycle  costs.  It  is  recognized,  not  all  9  domains  were  represented  in  the  framework  for  this 
effort,  as  many  have  a  more  direct  impact  on  the  cost,  schedule,  and  risk  objectives 
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described  above.  Further  evaluation  and  investigation  could  be  perfonned  to  more  fully 
incorporate  additional  HSI  impacts  into  a  larger  HMD  evaluation  framework  applied 
across  all  the  acquisition  objectives. 

Account  for  Unique  HMD  Capabilities 

As  with  most  technology  options,  each  HMD  system  brings  their  own  unique 
integration  approach  and  capabilities.  For  example,  some  systems  have  full  color 
symbology  while  others  produce  monochrome  displays.  One  solution  may  feature  a 
biodynamic  sensor  that  measures  the  pilot’s  blood  oxygen  levels,  and  another  may 
incorporate  an  ambient  light  sensor  that  automatically  drives  a  change  in  display 
brightness  as  the  environment  changes.  For  this  effort,  it  was  assumed  that  the 
pilot/maintainer  feedback  forms  may  account  for  some  of  the  additional  capabilities  not 
defined  in  requirements  as  they  would  likely  add  to  the  usability  or  satisfaction  of  the 
system.  However,  it  is  recognized  and  desired  that  future  research  should  detennine  a 
means  of  quantitatively  incorporating  unique  capabilities  into  the  value  metrics.  This 
would  allow  for  added  system  value  with  an  increase  in  capabilities  and  therefore  a  more 
thorough  assessment  of  alternatives. 

Breakout  Mission  Tasks  and  Environments 

Another  opportunity  for  adding  tailorability  to  this  framework  would  be  to 
delineate  Task/Environment  bins  based  upon  the  multiple  mission  types  flow  by  the 
aircraft.  A  night  mission  would  likely  have  different  objectives  needs  and  consequently 
HMD  capability  requirements  than  a  day  mission.  Therefore,  each  can  be  broken  out  as 
its  own  value  hierarchy.  This  could  create  multiple  perfonnance  type  hierarchies  (such  as 
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Day,  Night,  Air-to-Air,  and  Air-to-Ground)  under  the  HMD  Value,  Impact  on 
Operational  Effectiveness  score.  The  evaluation  measure  collection  itself  may  not 
change,  but  the  value  function  curves  and  preference  weights  may  vary  based  on  mission 
type.  Each  hierarchy  would  have  its  own  weight  to  represent  either  importance  of  the 
task/environment  type  or  the  proportion  at  which  that  mission  type  is  flown  over  the 
others. 

Conclusions  of  Research 

This  research  effort  has  shown  the  development  and  utility  of  a  value  based 
quantitative  assessment  framework  for  helmet  mounted  display  system  alternatives  that 
incorporates  critical  Human  System  Integration  principles.  The  usability  of  this  model 
comes  from  its  structure  and  inherent  tailorability,  which  can  allow  for  different  types  of 
technology  to  be  evaluated  on  varying  aircraft  platfonns.  The  insight  offered  by 
perfonning  an  analysis  such  as  this  can  provide  decision  makers  with  defendable, 
subjective  information  for  alternative  selection.  Future  research  can  enhance  the  basic 
framework  by  refining  evaluation  metrics  and  expanding  the  Value  Focused  Thinking 
methodology  across  the  crosscutting  military  acquisition  priorities. 
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Appendix  A  :  Initial  HMD  Factor  List 


Weight 

Anthropometric  Accommodation 

C.G. 

User  adjustability 

Display  Resolution 

Growth  Capability 

Contrast  at  worst  ambient  condition 

Mx  Impacts 

Total  System/Display  Refresh  Rate 

MTBF 

Display  Luminance 

MTBR 

Display  Color 

MTBME 

Symbology  Sets 

Availability 

Ocularity 

training  required 

Potential  for  optical  issues  (jitter,  ghosting, 
etc.) 

support  equipment  required 

F.O.V 

facilities/tooling  required 

NVIS/LEP  compatibility 

service  life 

Latency 

cleaning 

Accuracy 

aircrew  flight  equipment  compatibility 

Drift 

dimming  range 

Noise  Attenuation 

reflections 

Sound  Usage 

power  required 

Mx  Burden/Reliability 

Ocular  Rivalry 

Ease  of  Use/Pilot  Interface 

Pilot  Control  Interface 

User  Fit/Comfort 

Ease  of  Calibration 

A/C  Integration 

Built  in  Test 

Additional  Capabilities 

Environmental  compatibility 

Escape  Impact 

Amount  of  symbology 

Vibration 

Display  tailoring 

Effect  on  External  Viewing 

Display  Blanking/Declutter 

Angular  Resolution 

Dimming  Range 

Graphics  Processing 

Display  uniformity 

Stability  (symb,  tracking) 

Display  Recording 

Display  Distortion 

Compatibility  with  A/C  and  environment  lighting 

Exit  Pupil 

Helmet/visor  protection 

Eye  Relief 

Failure  risks 

Modular  Transfer  Function 

Personnel  required  for  maintenance 

Volume  of  Space  Required  (size) 

Security 

Frangibility 

Technology  Readiness  Level 
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Appendix  B  :  Questionnaires 


HMD-Pilot  Feedback  Questionnaire 


Organization:. 


HMD  Analysis  Questionnaire  -  Pilot 
Pilot: _  Date: _  HMD  System:. 


1 

2 

3 

4 

Completely 
Unacceptable, 
Needs  Major 
Improvements 

Unacceptable, 
Needs  Minor 
Improvements 

Minimally 

Acceptable 

Completely 

Acceptable 

While  wearineone  of  the  Helmet  Mounted  Display  alternatives,  please  rate  the  followingfeatures. 

Note:  Please  odd  comments  or  observations  for  system  improvements. 


1.  Fit  and  Comfort: 


a)  The  ability  to  Don/Doffthe  HMD 

b)  The  ability  to  change  between  DAY and  NIGHTHMD  configurations 

c)  Overall  weight 

d)  Center  of  Gravity -ability  to  move  head  as  necessary,  impact  on  fatigue,  etc. 
e|  Comfort  compared  to  legacy  helmet 

f)  The  ability  to  perform  normal  aircrew  duties  with  DAY  configuration,  i.e.  checklists,  check-6 

g)  The  abilityto  perform  normal  aircrew  duties  with  NIGHT  configuration,  i.e.  checklists,  check-6 

Comments: _ 


12  3  4 
12  3  4 
12  3  4 
12  3  4 
12  3  4 
12  3  4 
12  3  4 


2.  Observe  the  HMD  display  and  provide  a  rating: 

a)  Readability  and  usability  ofthe  display  symbology 

b)  Maximum  brightness  of  display 

c)  Size  of  display 

d)  The  ability  ofthe  HMD  to  be  confidently  used  as  required  for  safety  of  flight 
gr^mission  completion 

Comments: _ 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


3.  Observe  color,  contrast,  and  visibility  of  surfaces  throughout  the  cockpit  and  provide  a  rating.  These  include; 
switches,  printed  words  and  symbols,  as  well  as  kneeboards,  manuals,  and  otherapplicable  materials: 

a)  Readability  and  usability  ofsymbologyon  surfaces  12  3  4 

b)  Distinguishingcontrastorcolorchangesofsurfaces  12  3  4 

c)  The  abilityto  confidently  use  commandsand  instructionson  surfaces  for  safety  of  flight  12  3  4 

gr^mission  completion 

Comments: _ 
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HMD-Pilot  Feedback  Questionnaire 


5.  Observe  interior  aircraft  lighting  and  provide  a  rating: 

a)  Distinguishing  illumination  of  controls  and  displays  12  3  4 

b)  The  visibility  of  interior  lightingfor  safety  offlight  and  mission  completion  12  3  4 

Comments: _ 


6.  Observe  exterior  aircraft  lights  and  airfield  lighting  and  provide  a  rating: 

a)  The  visibility  and  discernibility  of  exterior  lights 

b)  Distinguishing  color  or  brightness  changes  of  exterior  lights 

c}  The  visibility  of  exterior  lighting  for  safety  offlight  and  mission  completion 
Comments: _ 


12  3  4 
12  3  4 
12  3  4 


7.  Observe  the  visibility  of  the  scene  outside  the  aircraft  and  providea  rating: 

a)  Discernibility  ofobjectsand  geography  in  the  external  scene  12  3  4 

b)  Visibility  through  canopy  12  3  4 

kg.  No  abnormal  reflections,  glare,  or  image  distortion  that  would  interfere  with 
jgfgjy. of  flight  and  mission  completion 

c|  Effect  on  expected  field  of  view  12  3  4 

Com  me  nts: _ 


8.  Integration: 

a)  The  ability  to  adjust  HMD  display  brightnessto  an  appropriate  night  level  12  3  4 

b|  The  ability  to  turn  offthe  HMD  display  12  3  4 

c|  The  ability  to  align/calibrate  HMD  symbology  12  3  4 

Comments: _ 


Additional  Feedback: 


110 


HMD_  Main  tamer  Feedback  Questionnaire 


HMD  Analysis  Questionnaire 

Aircraft  Maintainer: _  Life  Support  Back  shop  Maintainer: _  Date:_ 

HMD  System: _ 


1 

2 

3 

4 

Completely 
Unacceptable, 
Needs  Major 
Improvements 

Unacceptable, 
Needs  Minor 
Improvements 

Minimally 

Acceptable 

Completely 

Acceptable 

While  evaluating  one  of  the  Helmet  Mounted  Display  alternatives,  please  rate  the  folowing  features. 

Vote:  D.'ease  add  comments  or  absw.ations  for  system  improvements. 

1.  Deconstruction:  (8) 

a)  The  amount  of  steps  required  to  deconstruct  HMO  components  1  2  3  A 

b)  The  amount  of  tooling  required  to  deconstruct  HMD  12  3  4 

Comments: _ 


2.  Construction:  (8) 

a|  The  amount  of  steps  required  to  construct  HMD  components  12  3  4 

b)  The  amount  of  toobig  required  to  deconstruct  HMD  12  3  4 

Comments: _ 


3.  Pre-fSght  (8) 

a)  The  amount  of  steps  required  to  fit  and  check  HMD  12  3  4 

b)  The  complication  of  pre-fight  tasks  12  3  4 

Comments: _ 


5.  Post-fight:  (8] 

a)  The  amount  of  steps  required  for  sto-age  of  HMD  12  3  4 

b)  The  compfeation  of  sto-age  tasks  12  3  4 

Comments: _ 


6.  On-Aircraft  Servicing:  (12) 

a)  The  amount  of  toofcng  required 

b)  The  amount  of  steps  required  to  completely  remove  and  replace  HMD  components 

c)  The  amount  of  cockpit  hardware  to  be  removed  for  regular  maintenance 


12  3  4 
12  3  4 
12  3  4 
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HMD-Pilot  Feedback Quesbonm 


S.  Observe  interior  aircraft  lighting  and  provide  a  rating; 

a}  Distinguishing  Summation  of  controls  and  displays  12  3  4 

b)  The  vis  S  Sty  of  interior  ighting  for  safety  of  fight  and  mission  completion  12  3  4 

Comments: _ 


6.  Observe  exterior  aircraft  lights  and  airfield  lighting  and  provide  a  rating; 

a|  The  visSSty  and  discemSSty  of  exterior  lights 

b)  Distxigucshxig  color  or  brightness  changes  of  exterior  lights 

C)  The  visSSty  of  exterior  lighting  for  safety  of  fSght  and  mission  completion 

Comments ; _ 


12  3  4 
12  3  4 
12  3  4 


7.  Observe  the  visSSty  of  the  scene  outside  the  aircraft  and  provide  a  rating; 

a)  D  cscemibSty  of  ablets  and  geography  in  the  external  scene 

b)  Vis  A  Sty  through  canopy 

ie  Ho  abnormal  reflections  glare,  or  Image  distortion  that  would  interfere  with 
safety  of  fSght  and  mission  completion 

c)  Effect  on  expected  field  of  view 

Comments: _ 


12  3  4 
12  3  4 

12  3  4 


8.  Integration: 

a)  The  abSty  to  adjust  HMD  display  brightness  to  an  appropriate  night  level  12  3  4 

b)  The  abSty  to  turn  off  the  HMO  display  12  3  4 

c)  The  abSty  to  afign/caSbrate  HMD  symbology  12  3  4 

Comments: _ 


Additional  Feedback: 
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